-- Logs begin at Wed 2022-09-28 07:58:15 UTC, end at Wed 2022-09-28 09:04:21 UTC. -- Sep 28 08:23:19.770028 dv-cinder-844563-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 28 08:23:19.782131 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 28 08:23:19.784031 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 28 08:23:19 2022] *** Sep 28 08:23:19.784031 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 28 08:23:19.784031 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 28 08:23:19.784031 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: nodename: dv-cinder-844563-8-iscsi Sep 28 08:23:19.784031 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: machine: x86_64 Sep 28 08:23:19.784031 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: clock source: unix Sep 28 08:23:19.784031 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: pcre jit disabled Sep 28 08:23:19.784031 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: detected number of CPU cores: 4 Sep 28 08:23:19.784031 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: current working directory: / Sep 28 08:23:19.784031 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: detected binary path: /usr/bin/uwsgi-core Sep 28 08:23:19.784613 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: your processes number limit is 47680 Sep 28 08:23:19.784613 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: your memory page size is 4096 bytes Sep 28 08:23:19.784613 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: detected max file descriptor number: 2048 Sep 28 08:23:19.784613 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: lock engine: pthread robust mutexes Sep 28 08:23:19.784613 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: thunder lock: enabled Sep 28 08:23:19.784613 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 28 08:23:19.784613 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 28 08:23:19.824852 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: Python main interpreter initialized at 0x55c07f5e99c0 Sep 28 08:23:19.824852 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: python threads support enabled Sep 28 08:23:19.824852 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: your server socket listen backlog is limited to 100 connections Sep 28 08:23:19.824852 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: your mercy for graceful operations on workers is 90 seconds Sep 28 08:23:19.825292 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: mapped 403077 bytes (393 KB) for 2 cores Sep 28 08:23:19.825373 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: *** Operational MODE: preforking *** Sep 28 08:23:19.825624 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: *** uWSGI is running in multiple interpreter mode *** Sep 28 08:23:19.825624 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: spawned uWSGI master process (pid: 81471) Sep 28 08:23:19.825839 dv-cinder-844563-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 28 08:23:19.829142 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: spawned uWSGI worker 1 (pid: 81482, cores: 1) Sep 28 08:23:19.829142 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: spawned uWSGI worker 2 (pid: 81483, cores: 1) Sep 28 08:23:19.829142 dv-cinder-844563-8-iscsi devstack@keystone.service[81471]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 28 08:23:24.266511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 08:23:24.266511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: from cryptography.utils import int_from_bytes Sep 28 08:23:24.267318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 08:23:24.267318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: from cryptography.utils import int_from_bytes Sep 28 08:23:24.288597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 08:23:24.288597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: from cryptography.utils import int_from_bytes Sep 28 08:23:24.289237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 08:23:24.289237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: from cryptography.utils import int_from_bytes Sep 28 08:23:24.320026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 28 08:23:24.320211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 28 08:23:24.320395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] command line args: [] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 28 08:23:24.320667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 28 08:23:24.320847 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ================================================================================ {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 28 08:23:24.321124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] admin_token = **** {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.321385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] config_dir = [] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.321575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.321758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] config_source = [] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.321937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] debug = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.322137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.323437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.323437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.323437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.323437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.323437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] list_limit = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.323898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] log_config_append = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.323898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.323898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] log_dir = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.324196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] log_file = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.324269 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.324490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.324741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.324992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.325201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.325383 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.325630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.325885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.326121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.326350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.326586 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.326816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.327080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.327318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.327595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.328123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.328283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] publish_errors = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.328517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.328750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.328931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.329207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.329450 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.329708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] standard_threads = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.330437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.330682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.330861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.331063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] use_journal = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.331219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] use_json = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.331397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] use_stderr = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.331571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] use_syslog = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.331805 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.332070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.332312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.332576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.332820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.333073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.333343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.333655 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.334439 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.334439 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.334439 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.334650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.334850 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.335028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.335268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.335518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.335775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.336014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.336272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.336530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.336782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.337023 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.337226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 28 08:23:24.337308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.337423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 28 08:23:24.337559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.337636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] command line args: [] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 28 08:23:24.337817 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.337893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 28 08:23:24.338036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ================================================================================ {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 28 08:23:24.338119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.338291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] admin_token = **** {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.338375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.338523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.338597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] config_dir = [] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.338710 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.338791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.338893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] config_source = [] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.338973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.339073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] debug = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.339225 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.339302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.339476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.339554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.339668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.339742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.339917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.340003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.340163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.340244 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.340397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] list_limit = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.340480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.340577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] log_config_append = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.340653 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.340754 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.340874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.340950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] log_dir = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.341151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] log_file = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.341231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.341353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.341429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.341589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.341664 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.341823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.341907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.342057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.342132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.342232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.342308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.342392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.342508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.342637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.342748 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.342881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.342998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.343113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.343233 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.343343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.343471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.343576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.343705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.343791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.343944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.344025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.344180 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.344261 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.344431 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.344535 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.344663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.344901 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.345036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.345214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.345289 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] publish_errors = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.345389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.345609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.345688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.346334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.346411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.346514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.346623 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.346763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.346877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.346996 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.347117 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.347229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] standard_threads = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.347358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.347460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.347593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.347702 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.347860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.347937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.348041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] use_journal = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.348121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.348217 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] use_json = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.348321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.348405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] use_stderr = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.348555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] use_syslog = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.348637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.348785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 08:23:24.348864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.349044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.349153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.349311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.349388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.349561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.349635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.349798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.349880 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.350037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.350118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.350273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.350347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.350515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.350597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.350751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.350827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.350992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.351066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.351232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.351310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.351477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.351551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.351711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.351788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.351959 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.352040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.352194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.352269 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.352433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.352507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.352672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.352749 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.352907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.352988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.353166 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.353241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.354135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.354221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.354221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.354325 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.354325 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.354431 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.354431 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.354631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.354715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.354816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.354894 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.355010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.355137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.355250 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.355362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.355478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.355591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.355702 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.355822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.355935 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.356046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.356164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.356276 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.356395 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.356518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.356643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.356757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.356867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.356985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.357126 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.357251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.357376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.357489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.357590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.357704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.357815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.357927 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.358045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.358154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.358272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.358272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.358482 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.358482 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] auth.external = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.358698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.358698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.358900 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.359011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.359122 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.359234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.359353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.359472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.359584 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] auth.password = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.359696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.359808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] auth.token = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.359920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.360041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.360159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.360262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.360372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.360483 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.360594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.360713 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.360826 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.360937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.361053 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.361212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.361327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.361447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.361561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.361672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] credential.caching = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.361783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.361895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.362005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.362129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.362242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.362353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.362463 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.362574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.362693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.362806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.362916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.363034 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.363146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.363265 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.363452 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.363574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.363695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.363808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.363920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.364046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.364160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.364160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.364388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.364542 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.364662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.364775 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.364886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.364997 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.365188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.365305 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.365424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.365558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.365677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] federation.caching = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.365789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.365929 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.366043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.366156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.366267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.366378 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.366515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.366638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.366752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.366872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.366985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.367102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.367216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.367328 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.367516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.367642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.367757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.367859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.367970 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.368082 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.368202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.368323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.368442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.368556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.caching = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.368666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.368778 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.368889 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.369009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.369145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.369262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.369374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.369484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.369580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.369700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.369813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.369927 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.370038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.370471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.370592 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.370719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.370834 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.370946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.371061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.371183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.371296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.371417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.371531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.371650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.371764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.371882 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.371995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.372115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.372229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.372342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.372455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.372573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.372687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.372687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.372908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.373022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.373174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.373289 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.373401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.373527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.373642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.373754 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.373872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.373985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.374097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.374097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.374317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.374414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.374526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.374639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.374758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] auth.external = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.374758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.374971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.374971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.375181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.375294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.375414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.375529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.375642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] auth.password = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.375642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] auth.token = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.375852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.375965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.376085 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.376211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.376324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.376436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.376548 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.376661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.376857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.376969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.377067 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.377183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.377280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.377375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.377502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.377619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] credential.caching = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.377731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.377731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.377939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.378053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.378176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.378176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.378390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.378502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.378614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.378728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.378872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.378988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.379100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.379213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.379325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.379436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.379556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.379671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.379784 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.379897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.380009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.380128 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.380248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.380367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.380480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.380592 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.380705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.380817 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.380937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.381055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.381187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.381301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] federation.caching = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.381413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.381525 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.381649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.381763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.381877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.381989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.382101 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.382219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.382341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.382454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.382567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.382680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.382791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.382905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.383024 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.383138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.383234 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.383329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.383423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.383517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.383619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.383734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.383846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.383960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.caching = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.384078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.384192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.384313 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.384428 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.384540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.384653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.384765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.384884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.384999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.385193 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.385321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.385435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.385548 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.385672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.385787 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.385899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.386013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.386125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.386238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.386358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.386471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.386583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.386696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.386807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.386920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.387042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.387158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.387270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.387382 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.387494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.387606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.387726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.387840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.387952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.388064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.388064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.388276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.388276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.388497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.388609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.388722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.388834 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.388947 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.389068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.389202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] resource.caching = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.389315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.389428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.389540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.389662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.389784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.389899 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.390011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.390124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.390237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.390349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.390473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.390587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.390700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.390812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.390928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.391041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.391161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.391275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.391388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.391499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] role.caching = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.391611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.391724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] role.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.391844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.391958 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.392071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.392183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.392296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.392961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.393189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.393310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.393424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.393537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.393650 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.393762 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.393887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.394002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.394114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.394227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.394339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.394453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.394574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.394688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.394801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.394914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.395026 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.395138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.395259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.395373 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.395471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.395583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.395696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.395696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.395917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.395917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.396127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.396239 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.396352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.396472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.396588 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.396700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.396813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.396925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.397037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.397230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.397347 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.397459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.397574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.397686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.397798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.397918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.398032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.398145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.398257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.398369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.398481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.398634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.398750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.398861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.398973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.399083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.399230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.399230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.399777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.399777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.399990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.400110 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.400209 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.400329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.400442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.400555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] token.caching = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.400666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.400780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.400895 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.401017 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.401143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.401258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.401370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.401482 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.401595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.401595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.401822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.401941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.402055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.402167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.402279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.402399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.402518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.402631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.402743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.402857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.402969 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.403090 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.403204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.403322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.403436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.403548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.403660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.403780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.403895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.404008 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.404119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.404240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.404361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.404475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.404599 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.404715 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.404827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.connection = **** {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.404930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.405041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.405175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] resource.caching = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.405300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.405421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.405535 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.405648 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.405760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.405994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.406108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.406221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.406333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.406445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.406557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.406677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.406792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.406910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.407025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.407395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.407513 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.407635 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.407749 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] role.caching = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.407862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.407975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] role.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.408153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.408760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.408887 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.409004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.409134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.409262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.409378 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.409491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.409612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.409727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.409839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.409951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.410064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 28 08:23:24.410178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.410178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.410178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.410178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.410178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.410703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.410703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.410928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.411297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.411537 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.411789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.412040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.412291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.412545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.412781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.413037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.413316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.413556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.413789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.414020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.414251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.414504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.414759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.415005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.415233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.415469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.415693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.415922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] token.caching = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.416158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.416402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.416639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.416895 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.417166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.417408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.417653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.417898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.418146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.418381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.418620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.418854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.419099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.419378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.419608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.419851 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.420115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.420373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.connection = **** {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.420616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.420862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.421122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.421368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.421604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.421839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.422074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.422307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.422561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.422796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.423040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.423277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.423522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.423820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.424067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.424297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.424539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.424776 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 08:23:24.424948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81483) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 28 08:23:24.438506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.438735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.438965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.447542 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.450168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.450401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.450623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.459681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.476735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.476960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.478461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.478689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.533888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.542472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.585844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.586086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.590750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.590970 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.745317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.745549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.745722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.745814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.745958 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.746045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.746320 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.746423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.746501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.746587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.746587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.746785 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.771023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.771208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.771291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.771379 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.771446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.771597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.771676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.771825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.771918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.772048 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.772130 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.772265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.772345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.772481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.784298 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.784534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.797537 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.797782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.797975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.798195 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.798349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.798490 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.798627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.798762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.798966 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.799103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81483) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.799237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.799237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.799484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.799629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 08:23:24.816510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.816708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.816708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 08:23:24.816914 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 08:23:24.817046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.817219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.817300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 08:23:24.817399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 08:23:24.817532 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.817618 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.817792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.817874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.818054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.818151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.818314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.818394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.827989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.828104 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.828268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.828353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.828541 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.828641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.828823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.828904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.836544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.836735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.839523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.839674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.839801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.839927 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.840059 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.840187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.840342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.840437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.840604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.840687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.840865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.840984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.841146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.841232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.841425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.841529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.841698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.841799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.841974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.842060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.842243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.842322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.865680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.865869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.867178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.867275 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.867463 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.867582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.872154 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.872259 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.872437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.872521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.872719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.872801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.879747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.879882 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.880035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.880126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.886142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.886321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.886457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.886535 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.886726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.886835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.887013 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.887094 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.887292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.887371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.887574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.887657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.887847 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.887926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.888118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.888210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.904285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.904493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.904634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.904721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.908656 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.908846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.912038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.912448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.912448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.912650 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.916691 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.916842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.919922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.920048 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.923334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.923499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.924154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.924244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.924444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.924528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.924741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.924896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.925026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.925187 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.925319 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.925472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.942109 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.942374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.942519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.942655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.942794 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.942964 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.943098 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.943227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.950030 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.950227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.951607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.951772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.953181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.953296 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.953470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.953605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.956703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.956875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.957003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.957204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.957321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.957498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.957611 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.957784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.958019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.958192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.969457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.969656 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.969784 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.969865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.970042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.970153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.970325 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.970407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.971267 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.971358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.971561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.971658 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.971868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.971952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.987580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.987806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.990561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.990668 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.993522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.993645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.994965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.995074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.996465 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.996569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:24.996749 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.996835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.997208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:24.997347 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.003750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.003909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.005261 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:25.005438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:25.008170 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.008333 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.008572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.008693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.008916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.009045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.009195 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.009314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.016967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:25.017131 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:25.017294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.017381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.018188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.018290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.025427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81483) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:25.025672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 08:23:25.025811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.025918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.026007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.026086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.026217 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.026306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.026473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.026601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.026729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.026808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.027010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.027235 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.027327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.027399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.027598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.027685 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.027928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.028286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.028386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.028524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.028689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.028771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.028881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.029020 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.029173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81483) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.029402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 08:23:25.065342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c07f5e99c0 pid: 81483 (default app) Sep 28 08:23:25.065591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c07f5e99c0 pid: 81482 (default app) Sep 28 08:23:25.068786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60adc6c-3f3b-456b-9eb5-fab20358cc95 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:25.069074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60adc6c-3f3b-456b-9eb5-fab20358cc95 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:25.069372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60adc6c-3f3b-456b-9eb5-fab20358cc95 None None] PATH_INFO: `/v3` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:25.073601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Wed Sep 28 08:23:25 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:32.168561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c58937-fa43-48ac-82f8-aedd7703c751 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:32.169080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c58937-fa43-48ac-82f8-aedd7703c751 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:32.169080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c58937-fa43-48ac-82f8-aedd7703c751 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:32.173498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:23:32.179155 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07405407-c85f-4478-b85b-821116851771 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:32.179643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07405407-c85f-4478-b85b-821116851771 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:32.179779 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07405407-c85f-4478-b85b-821116851771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:32.250781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: INFO dbcounter [None req-07405407-c85f-4478-b85b-821116851771 None None] Registered counter for database keystone Sep 28 08:23:32.271375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG oslo_db.sqlalchemy.engines [None req-07405407-c85f-4478-b85b-821116851771 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81483) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 28 08:23:32.278015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writer thread running {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 28 08:23:32.459414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG passlib.handlers.bcrypt [None req-07405407-c85f-4478-b85b-821116851771 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81483) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 28 08:23:32.459810 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG passlib.handlers.bcrypt [None req-07405407-c85f-4478-b85b-821116851771 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81483) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 28 08:23:32.517627 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-07405407-c85f-4478-b85b-821116851771 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:32.519208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-07405407-c85f-4478-b85b-821116851771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:32.651359 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 28 08:23:33.099502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c58937-fa43-48ac-82f8-aedd7703c751 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:33.168555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: INFO dbcounter [None req-51c58937-fa43-48ac-82f8-aedd7703c751 None None] Registered counter for database keystone Sep 28 08:23:33.190949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG oslo_db.sqlalchemy.engines [None req-51c58937-fa43-48ac-82f8-aedd7703c751 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81482) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 28 08:23:33.197211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writer thread running {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 28 08:23:33.321248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c58937-fa43-48ac-82f8-aedd7703c751 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:33.325502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3eec3-b1f3-4d71-a61d-61f2e84dc0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:33.326143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3eec3-b1f3-4d71-a61d-61f2e84dc0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:33.326415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3eec3-b1f3-4d71-a61d-61f2e84dc0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:33.326683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3eec3-b1f3-4d71-a61d-61f2e84dc0e3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:33.335303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3eec3-b1f3-4d71-a61d-61f2e84dc0e3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:33.390024 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6ea3eec3-b1f3-4d71-a61d-61f2e84dc0e3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 28 08:23:33.390271 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3eec3-b1f3-4d71-a61d-61f2e84dc0e3 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:33.401340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Sep 28 08:23:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:23:33.407531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07405407-c85f-4478-b85b-821116851771 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:33.424597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07405407-c85f-4478-b85b-821116851771 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:33.429004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be261a24-89ec-42fa-9047-178fcab195e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:33.430150 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be261a24-89ec-42fa-9047-178fcab195e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:33.430432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be261a24-89ec-42fa-9047-178fcab195e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:33.430699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be261a24-89ec-42fa-9047-178fcab195e8 admin admin] PATH_INFO: `/v3/projects/c189a0d1d6b0443ba4af2cd3c46f5c43` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:33.432278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be261a24-89ec-42fa-9047-178fcab195e8 admin admin] RBAC: Authorizing `identity:get_project(project_id=c189a0d1d6b0443ba4af2cd3c46f5c43)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:33.485431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: WARNING keystone.common.rbac_enforcer.enforcer [None req-be261a24-89ec-42fa-9047-178fcab195e8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 28 08:23:33.485783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be261a24-89ec-42fa-9047-178fcab195e8 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:33.491027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Wed Sep 28 08:23:33 2022] GET /identity/v3/projects/c189a0d1d6b0443ba4af2cd3c46f5c43 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:23:35.681714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53024ec8-d6fa-4c1e-8d7e-8458da3cedf1 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:35.682281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53024ec8-d6fa-4c1e-8d7e-8458da3cedf1 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:35.682281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53024ec8-d6fa-4c1e-8d7e-8458da3cedf1 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:35.686364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:23:35.692510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee52037-b32d-430b-84d6-51c91da5b6ce None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:35.692973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee52037-b32d-430b-84d6-51c91da5b6ce None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:35.693063 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee52037-b32d-430b-84d6-51c91da5b6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:35.713489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-3ee52037-b32d-430b-84d6-51c91da5b6ce None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:35.715049 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3ee52037-b32d-430b-84d6-51c91da5b6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:35.723571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 28 08:23:35.996106 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53024ec8-d6fa-4c1e-8d7e-8458da3cedf1 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:36.008820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53024ec8-d6fa-4c1e-8d7e-8458da3cedf1 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:36.013079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb90b6-49ff-404c-9507-e8c2d7973ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:36.013705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb90b6-49ff-404c-9507-e8c2d7973ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:36.013989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb90b6-49ff-404c-9507-e8c2d7973ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:36.014266 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb90b6-49ff-404c-9507-e8c2d7973ea2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:36.022717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb90b6-49ff-404c-9507-e8c2d7973ea2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:36.024510 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb90b6-49ff-404c-9507-e8c2d7973ea2 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:36.034040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Sep 28 08:23:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:23:36.039971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee52037-b32d-430b-84d6-51c91da5b6ce None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:36.049343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee52037-b32d-430b-84d6-51c91da5b6ce None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:36.053666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c5054-066e-4c37-84dd-77bef34c7012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:36.054296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c5054-066e-4c37-84dd-77bef34c7012 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:36.054578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c5054-066e-4c37-84dd-77bef34c7012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:36.054853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c5054-066e-4c37-84dd-77bef34c7012 admin admin] PATH_INFO: `/v3/users/3300b0df5e5741c8b0d940d0202bb604` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:36.056180 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c5054-066e-4c37-84dd-77bef34c7012 admin admin] RBAC: Authorizing `identity:get_user(user_id=3300b0df5e5741c8b0d940d0202bb604)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:36.058060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c5054-066e-4c37-84dd-77bef34c7012 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:36.059623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Wed Sep 28 08:23:36 2022] GET /identity/v3/users/3300b0df5e5741c8b0d940d0202bb604 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:38.353105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518d566-0e58-4e9b-86d5-9f37ad5c9b7e None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:38.353651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518d566-0e58-4e9b-86d5-9f37ad5c9b7e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:38.353651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518d566-0e58-4e9b-86d5-9f37ad5c9b7e None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:38.355140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:38.356213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5583e-35b8-40ae-b25e-d07b6b963ca9 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:38.356623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5583e-35b8-40ae-b25e-d07b6b963ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:38.356758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5583e-35b8-40ae-b25e-d07b6b963ca9 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:38.358138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:23:38.360593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de6a27-f997-4988-b043-3917bcd9b695 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:38.361411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de6a27-f997-4988-b043-3917bcd9b695 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:38.361696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de6a27-f997-4988-b043-3917bcd9b695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:38.364187 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254023c0-d6f0-4c61-bfcb-7d05bc915fcd None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:38.364468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254023c0-d6f0-4c61-bfcb-7d05bc915fcd None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:38.364749 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254023c0-d6f0-4c61-bfcb-7d05bc915fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:38.388577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-254023c0-d6f0-4c61-bfcb-7d05bc915fcd None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:38.389835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-254023c0-d6f0-4c61-bfcb-7d05bc915fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:38.405691 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG passlib.handlers.bcrypt [None req-70de6a27-f997-4988-b043-3917bcd9b695 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81482) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 28 08:23:38.406095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG passlib.handlers.bcrypt [None req-70de6a27-f997-4988-b043-3917bcd9b695 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81482) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 28 08:23:38.455800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-70de6a27-f997-4988-b043-3917bcd9b695 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:38.457478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-70de6a27-f997-4988-b043-3917bcd9b695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:38.474908 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 7/12] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:23:38.566888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 6/13] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:23:38.752527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254023c0-d6f0-4c61-bfcb-7d05bc915fcd None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:38.785062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:38.835159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de6a27-f997-4988-b043-3917bcd9b695 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:38.880635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254023c0-d6f0-4c61-bfcb-7d05bc915fcd None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:38.922360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa19ff-fb39-4f37-aff0-bfac56777745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:38.923014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa19ff-fb39-4f37-aff0-bfac56777745 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:38.923301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa19ff-fb39-4f37-aff0-bfac56777745 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:38.923595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa19ff-fb39-4f37-aff0-bfac56777745 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:38.939759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa19ff-fb39-4f37-aff0-bfac56777745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-73fa19ff-fb39-4f37-aff0-bfac56777745 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:23:38.948922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:23:38.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:23:38.950990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 8/14] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:23:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:23:38.956881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa19ff-fb39-4f37-aff0-bfac56777745 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:38.957049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de6a27-f997-4988-b043-3917bcd9b695 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:38.959663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:38.999931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f72c47-b004-477e-9c70-bcd5d832e6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:39.004076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f72c47-b004-477e-9c70-bcd5d832e6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:39.004076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f72c47-b004-477e-9c70-bcd5d832e6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:39.004076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f72c47-b004-477e-9c70-bcd5d832e6d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:39.051924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:39.073716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa19ff-fb39-4f37-aff0-bfac56777745 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:39.076899 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f72c47-b004-477e-9c70-bcd5d832e6d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:39.078567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f72c47-b004-477e-9c70-bcd5d832e6d6 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:39.115726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41201d7-960c-46f7-9b8a-1f336301f991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:39.116361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41201d7-960c-46f7-9b8a-1f336301f991 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:39.116645 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41201d7-960c-46f7-9b8a-1f336301f991 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:39.116922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41201d7-960c-46f7-9b8a-1f336301f991 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:39.117652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41201d7-960c-46f7-9b8a-1f336301f991 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:39.117961 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41201d7-960c-46f7-9b8a-1f336301f991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:39.119397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41201d7-960c-46f7-9b8a-1f336301f991 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:39.132482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 9/15] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:23:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:23:39.138424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41201d7-960c-46f7-9b8a-1f336301f991 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:39.184323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 7/16] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 08:23:39.190888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f72c47-b004-477e-9c70-bcd5d832e6d6 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:39.193739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:39.245364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41201d7-960c-46f7-9b8a-1f336301f991 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:39.281512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2a2d9-de0a-40bf-ac97-12c8784ef9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:39.282065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2a2d9-de0a-40bf-ac97-12c8784ef9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:39.282353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2a2d9-de0a-40bf-ac97-12c8784ef9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:39.282631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2a2d9-de0a-40bf-ac97-12c8784ef9b8 None admin] PATH_INFO: `/v3/users/3300b0df5e5741c8b0d940d0202bb604` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:39.283815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2a2d9-de0a-40bf-ac97-12c8784ef9b8 None admin] RBAC: Authorizing `identity:get_user(user_id=3300b0df5e5741c8b0d940d0202bb604)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:39.285302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2a2d9-de0a-40bf-ac97-12c8784ef9b8 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:39.286902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 10/17] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:23:39 2022] GET /identity/v3/users/3300b0df5e5741c8b0d940d0202bb604 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:23:39.292434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2a2d9-de0a-40bf-ac97-12c8784ef9b8 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:39.307751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f72c47-b004-477e-9c70-bcd5d832e6d6 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:39.309665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:39.346293 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf7355-0385-4c7c-87d0-3206dc33250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:39.347100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf7355-0385-4c7c-87d0-3206dc33250f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:39.347493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf7355-0385-4c7c-87d0-3206dc33250f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:39.348013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf7355-0385-4c7c-87d0-3206dc33250f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:39.374152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf7355-0385-4c7c-87d0-3206dc33250f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:39.375990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf7355-0385-4c7c-87d0-3206dc33250f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-52cf7355-0385-4c7c-87d0-3206dc33250f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:23:39.396335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:23:39.397418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:23:39.398362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:23:39.398362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:23:39.398362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:23:39.398362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 8/18] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:23:39.405193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf7355-0385-4c7c-87d0-3206dc33250f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:39.414372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2a2d9-de0a-40bf-ac97-12c8784ef9b8 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:39.452145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef38d8-d129-44c9-b651-34440af29700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:39.452785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef38d8-d129-44c9-b651-34440af29700 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:39.453062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef38d8-d129-44c9-b651-34440af29700 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:39.453368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef38d8-d129-44c9-b651-34440af29700 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:39.454505 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef38d8-d129-44c9-b651-34440af29700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:39.456441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef38d8-d129-44c9-b651-34440af29700 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:39.458037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 11/19] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:23:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:23:39.463779 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef38d8-d129-44c9-b651-34440af29700 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:39.469229 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:39.523322 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf7355-0385-4c7c-87d0-3206dc33250f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:39.559759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:39.560168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c09ca-851b-4652-adbe-22175e150b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:39.560827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c09ca-851b-4652-adbe-22175e150b70 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:39.561134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c09ca-851b-4652-adbe-22175e150b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:39.561420 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c09ca-851b-4652-adbe-22175e150b70 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:39.562773 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c09ca-851b-4652-adbe-22175e150b70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:39.562773 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c09ca-851b-4652-adbe-22175e150b70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:39.564453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c09ca-851b-4652-adbe-22175e150b70 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:39.582003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef38d8-d129-44c9-b651-34440af29700 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:39.583480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 9/20] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:23:39.621809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d102f-b272-4c94-8581-b38b71229990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:39.622448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d102f-b272-4c94-8581-b38b71229990 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:39.622731 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d102f-b272-4c94-8581-b38b71229990 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:39.623007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d102f-b272-4c94-8581-b38b71229990 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:39.623789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d102f-b272-4c94-8581-b38b71229990 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.domain.id=default, user.id=3300b0df5e5741c8b0d940d0202bb604) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:39.624631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d102f-b272-4c94-8581-b38b71229990 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:39.625598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d102f-b272-4c94-8581-b38b71229990 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:39.633937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 12/21] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Sep 28 08:23:39 2022] GET /identity/v3/role_assignments?user.id=3300b0df5e5741c8b0d940d0202bb604&scope.domain.id=default&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 233 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:23:42.097691 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972d447-57c5-465e-a1c0-36ae0220afee None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:42.098228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972d447-57c5-465e-a1c0-36ae0220afee None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:42.098501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972d447-57c5-465e-a1c0-36ae0220afee None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:42.100124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 10/22] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:42.105361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6000607-a88f-4942-b44c-9bfca9751ee4 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:42.105798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6000607-a88f-4942-b44c-9bfca9751ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:42.105968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6000607-a88f-4942-b44c-9bfca9751ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:42.134183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-c6000607-a88f-4942-b44c-9bfca9751ee4 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:42.135423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c6000607-a88f-4942-b44c-9bfca9751ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:42.225054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 13/23] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:23:42.446869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed877ae2-7fe7-4d3c-886e-91da36be691e None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:42.447096 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed877ae2-7fe7-4d3c-886e-91da36be691e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:42.447480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed877ae2-7fe7-4d3c-886e-91da36be691e None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:42.448861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 11/24] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:23:42.460277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab45e8-d288-476d-9862-2e253920e2a0 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:42.460653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab45e8-d288-476d-9862-2e253920e2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:42.460884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab45e8-d288-476d-9862-2e253920e2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:42.494244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-e0ab45e8-d288-476d-9862-2e253920e2a0 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:42.495455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e0ab45e8-d288-476d-9862-2e253920e2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:42.516861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9edfec-5a98-4372-8ccc-2783e664b559 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:42.516861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9edfec-5a98-4372-8ccc-2783e664b559 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:42.516861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9edfec-5a98-4372-8ccc-2783e664b559 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:42.516861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 12/25] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:42.524244 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ebe1-88a0-4b3f-9d25-91e2da8e8230 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:42.524487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ebe1-88a0-4b3f-9d25-91e2da8e8230 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:42.525049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06ebe1-88a0-4b3f-9d25-91e2da8e8230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:42.555309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-dc06ebe1-88a0-4b3f-9d25-91e2da8e8230 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:42.556546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-dc06ebe1-88a0-4b3f-9d25-91e2da8e8230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:42.686625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 13/26] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:23:42.703733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 14/27] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:23:42.726645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06ebe1-88a0-4b3f-9d25-91e2da8e8230 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:42.794099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:42.900738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06ebe1-88a0-4b3f-9d25-91e2da8e8230 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:42.979517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593e168-6482-4e19-b3ec-8b1b8112ab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:42.981025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593e168-6482-4e19-b3ec-8b1b8112ab33 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:42.981833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593e168-6482-4e19-b3ec-8b1b8112ab33 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:42.982543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593e168-6482-4e19-b3ec-8b1b8112ab33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:43.013368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593e168-6482-4e19-b3ec-8b1b8112ab33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-d593e168-6482-4e19-b3ec-8b1b8112ab33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:23:43.032326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:23:43.034086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:23:43.035514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:23:43.035514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:23:43.035514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:23:43.035514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:23:43.035514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:23:43.035514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:23:43.035514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:23:43.035514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:23:43.035514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:23:43.035514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:23:43.035514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:23:43.035514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:23:43.040526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 14/28] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:23:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:23:43.047112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab45e8-d288-476d-9862-2e253920e2a0 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:43.069788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:43.128255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593e168-6482-4e19-b3ec-8b1b8112ab33 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:43.241348 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab45e8-d288-476d-9862-2e253920e2a0 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:43.287800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593e168-6482-4e19-b3ec-8b1b8112ab33 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:43.288921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b538d8-a1a2-45fc-af57-803b79419f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:43.289789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b538d8-a1a2-45fc-af57-803b79419f5d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:43.290070 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b538d8-a1a2-45fc-af57-803b79419f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:43.290346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b538d8-a1a2-45fc-af57-803b79419f5d None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:43.291950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b538d8-a1a2-45fc-af57-803b79419f5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:43.292260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b538d8-a1a2-45fc-af57-803b79419f5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:43.293724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b538d8-a1a2-45fc-af57-803b79419f5d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:43.296952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:43.310259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 15/29] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:23:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:43.311655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b538d8-a1a2-45fc-af57-803b79419f5d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:43.340557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b860ff3-8e9f-4c52-82e6-ed138c300ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:43.341281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b860ff3-8e9f-4c52-82e6-ed138c300ffc None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:43.341570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b860ff3-8e9f-4c52-82e6-ed138c300ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:43.342261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b860ff3-8e9f-4c52-82e6-ed138c300ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:43.355324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b860ff3-8e9f-4c52-82e6-ed138c300ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:43.356975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b860ff3-8e9f-4c52-82e6-ed138c300ffc None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-6b860ff3-8e9f-4c52-82e6-ed138c300ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:23:43.375738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:23:43.377247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:23:43.378027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:23:43.378027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:23:43.378027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:23:43.378027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 15/30] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:23:43.378921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b63de-e57a-468c-838b-634004e59b9d None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:43.378921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b63de-e57a-468c-838b-634004e59b9d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:43.379151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b63de-e57a-468c-838b-634004e59b9d None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:43.384577 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 16/31] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:43.384577 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b63de-e57a-468c-838b-634004e59b9d None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:43.402196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:43.450621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b538d8-a1a2-45fc-af57-803b79419f5d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:43.499217 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a8cfe-50b8-42f0-80a1-5050631c3b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:43.500671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a8cfe-50b8-42f0-80a1-5050631c3b0d None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:43.501533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a8cfe-50b8-42f0-80a1-5050631c3b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:43.502008 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a8cfe-50b8-42f0-80a1-5050631c3b0d None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:43.503516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a8cfe-50b8-42f0-80a1-5050631c3b0d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:43.505734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a8cfe-50b8-42f0-80a1-5050631c3b0d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:43.517110 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b63de-e57a-468c-838b-634004e59b9d None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:43.561858 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829466b7-5c58-4146-9d46-e6bb631db50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:43.562585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829466b7-5c58-4146-9d46-e6bb631db50f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:43.562865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829466b7-5c58-4146-9d46-e6bb631db50f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:43.563184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829466b7-5c58-4146-9d46-e6bb631db50f None admin] PATH_INFO: `/v3/users/3300b0df5e5741c8b0d940d0202bb604` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:43.564806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829466b7-5c58-4146-9d46-e6bb631db50f None admin] RBAC: Authorizing `identity:get_user(user_id=3300b0df5e5741c8b0d940d0202bb604)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:43.566349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829466b7-5c58-4146-9d46-e6bb631db50f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:43.572459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 17/32] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:23:43 2022] GET /identity/v3/users/3300b0df5e5741c8b0d940d0202bb604 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:43.572459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829466b7-5c58-4146-9d46-e6bb631db50f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:43.610259 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:43.668723 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 16/33] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 08:23:43 2022] POST /identity/v3/roles => generated 219 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:23:43.670147 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3130f-0f33-419f-9fb0-3f034524f5fb None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:43.670360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3130f-0f33-419f-9fb0-3f034524f5fb None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:43.670638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3130f-0f33-419f-9fb0-3f034524f5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:43.691281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-fbc3130f-0f33-419f-9fb0-3f034524f5fb None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:43.692913 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-fbc3130f-0f33-419f-9fb0-3f034524f5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:43.695125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829466b7-5c58-4146-9d46-e6bb631db50f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:43.735536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b826e-7275-44a5-a2a3-17b83e7c49a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:43.736585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b826e-7275-44a5-a2a3-17b83e7c49a0 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:43.737060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b826e-7275-44a5-a2a3-17b83e7c49a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:43.737609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b826e-7275-44a5-a2a3-17b83e7c49a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:43.738665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b826e-7275-44a5-a2a3-17b83e7c49a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:43.739167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b826e-7275-44a5-a2a3-17b83e7c49a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:43.740869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b826e-7275-44a5-a2a3-17b83e7c49a0 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:43.755210 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 18/34] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:43.756402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b826e-7275-44a5-a2a3-17b83e7c49a0 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:43.772419 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 17/35] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:23:43.773377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3130f-0f33-419f-9fb0-3f034524f5fb None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:43.894395 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:43.903474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:43.929890 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b826e-7275-44a5-a2a3-17b83e7c49a0 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:43.994173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3130f-0f33-419f-9fb0-3f034524f5fb None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:44.000908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9bd47-66ec-489e-9e3e-b03d08d22bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:44.002554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9bd47-66ec-489e-9e3e-b03d08d22bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:44.003105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9bd47-66ec-489e-9e3e-b03d08d22bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:44.003620 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9bd47-66ec-489e-9e3e-b03d08d22bc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:44.005029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9bd47-66ec-489e-9e3e-b03d08d22bc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:44.006809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9bd47-66ec-489e-9e3e-b03d08d22bc3 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:44.008645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 19/36] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:23:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:44.016807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9bd47-66ec-489e-9e3e-b03d08d22bc3 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:44.070004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757d225-98b4-469e-9530-ff12a0119802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:44.070758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757d225-98b4-469e-9530-ff12a0119802 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:44.071039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757d225-98b4-469e-9530-ff12a0119802 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:44.071311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757d225-98b4-469e-9530-ff12a0119802 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:44.071971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757d225-98b4-469e-9530-ff12a0119802 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:44.073751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757d225-98b4-469e-9530-ff12a0119802 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:44.122417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 18/37] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:23:43 2022] POST /identity/v3/projects => generated 301 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:23:44.123837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3845fe-5a7c-4af7-b3ce-ea8d4e9c27a8 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:44.124133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3845fe-5a7c-4af7-b3ce-ea8d4e9c27a8 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:44.124415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3845fe-5a7c-4af7-b3ce-ea8d4e9c27a8 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:44.133426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 19/38] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:44.139807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c295-4dad-405b-aca3-9d4c96baa692 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:44.140111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c295-4dad-405b-aca3-9d4c96baa692 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:44.140389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c295-4dad-405b-aca3-9d4c96baa692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:44.148744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9bd47-66ec-489e-9e3e-b03d08d22bc3 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:44.185619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-a670c295-4dad-405b-aca3-9d4c96baa692 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:44.187349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a670c295-4dad-405b-aca3-9d4c96baa692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:44.201337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b7079-9d2c-4d24-9f4c-b04b6c49ef95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:44.202436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b7079-9d2c-4d24-9f4c-b04b6c49ef95 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:44.202960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b7079-9d2c-4d24-9f4c-b04b6c49ef95 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:44.203459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b7079-9d2c-4d24-9f4c-b04b6c49ef95 None admin] PATH_INFO: `/v3/domains/default/users/3300b0df5e5741c8b0d940d0202bb604/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:44.206329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b7079-9d2c-4d24-9f4c-b04b6c49ef95 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3300b0df5e5741c8b0d940d0202bb604, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:44.208177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b7079-9d2c-4d24-9f4c-b04b6c49ef95 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:44.293497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 20/39] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:23:44.294308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670c295-4dad-405b-aca3-9d4c96baa692 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:44.346587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 20/40] 10.222.0.52 () {58 vars in 1361 bytes} [Wed Sep 28 08:23:44 2022] PUT /identity/v3/domains/default/users/3300b0df5e5741c8b0d940d0202bb604/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:23:44.348076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:44.349432 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08d30b-5a6e-4da9-9ee9-e489d1294c64 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:44.349903 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08d30b-5a6e-4da9-9ee9-e489d1294c64 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:44.350326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08d30b-5a6e-4da9-9ee9-e489d1294c64 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:44.356079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 21/41] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:44.362610 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dddcb9-ce1f-4fb6-a064-074734caa7c3 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:44.362942 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dddcb9-ce1f-4fb6-a064-074734caa7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:44.363148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dddcb9-ce1f-4fb6-a064-074734caa7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:44.383820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-a3dddcb9-ce1f-4fb6-a064-074734caa7c3 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:44.385475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a3dddcb9-ce1f-4fb6-a064-074734caa7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:44.400023 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:44.427815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670c295-4dad-405b-aca3-9d4c96baa692 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:44.475498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a21000-4710-4204-b7d1-711ae7dfccf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:44.476076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 22/42] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:23:44.476297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a21000-4710-4204-b7d1-711ae7dfccf6 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:44.476599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a21000-4710-4204-b7d1-711ae7dfccf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:44.476924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a21000-4710-4204-b7d1-711ae7dfccf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:44.477712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a21000-4710-4204-b7d1-711ae7dfccf6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:44.479152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a21000-4710-4204-b7d1-711ae7dfccf6 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:44.497349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 21/43] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 08:23:44 2022] POST /identity/v3/roles => generated 225 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:23:44.623985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dddcb9-ce1f-4fb6-a064-074734caa7c3 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:44.735095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dddcb9-ce1f-4fb6-a064-074734caa7c3 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:44.765417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a21000-4710-4204-b7d1-711ae7dfccf6 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:44.774469 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63247e0f-7386-4673-b1a5-a3032924c843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:44.775340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63247e0f-7386-4673-b1a5-a3032924c843 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:44.775621 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63247e0f-7386-4673-b1a5-a3032924c843 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:44.775894 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63247e0f-7386-4673-b1a5-a3032924c843 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:44.776572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63247e0f-7386-4673-b1a5-a3032924c843 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:44.778630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63247e0f-7386-4673-b1a5-a3032924c843 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:44.797509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 23/44] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 08:23:44 2022] POST /identity/v3/roles => generated 223 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:23:44.990392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:44.995212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a21000-4710-4204-b7d1-711ae7dfccf6 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:45.036618 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b610767-cd7b-47f8-9952-0cb33a26689b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:45.037314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b610767-cd7b-47f8-9952-0cb33a26689b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:45.037596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b610767-cd7b-47f8-9952-0cb33a26689b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:45.037876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b610767-cd7b-47f8-9952-0cb33a26689b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:45.038989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b610767-cd7b-47f8-9952-0cb33a26689b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:45.040439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b610767-cd7b-47f8-9952-0cb33a26689b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:45.042041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 22/45] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:23:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:23:45.047595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63247e0f-7386-4673-b1a5-a3032924c843 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:45.051597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:45.153015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63247e0f-7386-4673-b1a5-a3032924c843 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:45.186870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130b061-0a04-4d0a-8677-25c2668ba4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:45.187486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130b061-0a04-4d0a-8677-25c2668ba4ad None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:45.187772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130b061-0a04-4d0a-8677-25c2668ba4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:45.188049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130b061-0a04-4d0a-8677-25c2668ba4ad None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:45.188717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130b061-0a04-4d0a-8677-25c2668ba4ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:45.190239 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130b061-0a04-4d0a-8677-25c2668ba4ad None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:45.218536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 24/46] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:23:45 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:23:46.624073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82db6c-cefe-4abd-9817-221d10a4bc19 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:46.624580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82db6c-cefe-4abd-9817-221d10a4bc19 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:46.624662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82db6c-cefe-4abd-9817-221d10a4bc19 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:46.626021 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 23/47] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:46.631443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be085a23-b200-462f-9718-d909f728e048 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:46.631876 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be085a23-b200-462f-9718-d909f728e048 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:46.632008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be085a23-b200-462f-9718-d909f728e048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:46.652145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-be085a23-b200-462f-9718-d909f728e048 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:46.653388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-be085a23-b200-462f-9718-d909f728e048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:46.705538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:46.723607 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 25/48] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:23:46.992757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82db6c-cefe-4abd-9817-221d10a4bc19 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:47.098846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82db6c-cefe-4abd-9817-221d10a4bc19 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:47.132203 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab31a9f-b43b-4cc7-a43c-7565aeb5130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:47.132828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab31a9f-b43b-4cc7-a43c-7565aeb5130c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:47.133129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab31a9f-b43b-4cc7-a43c-7565aeb5130c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:47.133409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab31a9f-b43b-4cc7-a43c-7565aeb5130c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:47.144083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab31a9f-b43b-4cc7-a43c-7565aeb5130c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-8ab31a9f-b43b-4cc7-a43c-7565aeb5130c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:23:47.149038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:23:47.150373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:23:47.153906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 24/49] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:23:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:23:47.156390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be085a23-b200-462f-9718-d909f728e048 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:47.257306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be085a23-b200-462f-9718-d909f728e048 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:47.290810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ce416-40bb-4ad7-9149-2f95bdeda036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:47.291424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ce416-40bb-4ad7-9149-2f95bdeda036 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:47.291708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ce416-40bb-4ad7-9149-2f95bdeda036 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:47.291988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ce416-40bb-4ad7-9149-2f95bdeda036 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:47.292707 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ce416-40bb-4ad7-9149-2f95bdeda036 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:47.292976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ce416-40bb-4ad7-9149-2f95bdeda036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:47.294432 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ce416-40bb-4ad7-9149-2f95bdeda036 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:47.308793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 26/50] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:23:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:47.316107 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab31a9f-b43b-4cc7-a43c-7565aeb5130c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:47.346659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:47.420022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab31a9f-b43b-4cc7-a43c-7565aeb5130c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:47.422580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21709e8-1bf7-4a43-b4a8-568c15560a78 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:47.422869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21709e8-1bf7-4a43-b4a8-568c15560a78 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:47.423686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21709e8-1bf7-4a43-b4a8-568c15560a78 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:47.425029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 27/51] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:23:47.430059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c835d3f-aa16-4d8f-92ef-1c3abcfa11b7 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:47.430346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c835d3f-aa16-4d8f-92ef-1c3abcfa11b7 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:47.430626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c835d3f-aa16-4d8f-92ef-1c3abcfa11b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:47.456620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c3618-6c34-4865-a51e-c7d2eecb9a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:47.457312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c3618-6c34-4865-a51e-c7d2eecb9a6d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:47.457599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c3618-6c34-4865-a51e-c7d2eecb9a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:47.457879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c3618-6c34-4865-a51e-c7d2eecb9a6d None admin] PATH_INFO: `/v3/users/3300b0df5e5741c8b0d940d0202bb604` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:47.459082 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c3618-6c34-4865-a51e-c7d2eecb9a6d None admin] RBAC: Authorizing `identity:get_user(user_id=3300b0df5e5741c8b0d940d0202bb604)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:47.460600 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c3618-6c34-4865-a51e-c7d2eecb9a6d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:47.462522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 25/52] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:23:47 2022] GET /identity/v3/users/3300b0df5e5741c8b0d940d0202bb604 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:47.468286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c3618-6c34-4865-a51e-c7d2eecb9a6d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:47.552997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2c835d3f-aa16-4d8f-92ef-1c3abcfa11b7 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:47.554246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2c835d3f-aa16-4d8f-92ef-1c3abcfa11b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:47.579478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:47.591556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c3618-6c34-4865-a51e-c7d2eecb9a6d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:47.602843 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:47.627286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 28/53] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:23:47.627475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16c946-d04c-4b7e-aba6-6431952d82b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:47.627908 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16c946-d04c-4b7e-aba6-6431952d82b9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:47.628175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16c946-d04c-4b7e-aba6-6431952d82b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:47.628426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16c946-d04c-4b7e-aba6-6431952d82b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:47.629616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16c946-d04c-4b7e-aba6-6431952d82b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:47.631074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16c946-d04c-4b7e-aba6-6431952d82b9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:47.632653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 26/54] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:23:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:23:47.638326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c835d3f-aa16-4d8f-92ef-1c3abcfa11b7 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:47.737250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c835d3f-aa16-4d8f-92ef-1c3abcfa11b7 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:47.771347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30025cde-8cec-42e6-8bfd-f5f0ad71043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:47.771957 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30025cde-8cec-42e6-8bfd-f5f0ad71043e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:47.772241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30025cde-8cec-42e6-8bfd-f5f0ad71043e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:47.772517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30025cde-8cec-42e6-8bfd-f5f0ad71043e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:47.773279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30025cde-8cec-42e6-8bfd-f5f0ad71043e None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.domain.id=default, user.id=3300b0df5e5741c8b0d940d0202bb604) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:47.773594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30025cde-8cec-42e6-8bfd-f5f0ad71043e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:47.775328 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30025cde-8cec-42e6-8bfd-f5f0ad71043e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:47.783087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 29/55] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Sep 28 08:23:47 2022] GET /identity/v3/role_assignments?user.id=3300b0df5e5741c8b0d940d0202bb604&scope.domain.id=default&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:47.893505 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16c946-d04c-4b7e-aba6-6431952d82b9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:48.007159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16c946-d04c-4b7e-aba6-6431952d82b9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:48.041278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a17775-9bec-4e20-8f63-5f2444f677aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:48.041853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a17775-9bec-4e20-8f63-5f2444f677aa None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:48.042137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a17775-9bec-4e20-8f63-5f2444f677aa None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:48.042411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a17775-9bec-4e20-8f63-5f2444f677aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:48.043497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a17775-9bec-4e20-8f63-5f2444f677aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:48.044944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a17775-9bec-4e20-8f63-5f2444f677aa None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:48.046550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 27/56] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:23:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:48.052279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30025cde-8cec-42e6-8bfd-f5f0ad71043e None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:48.095925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:48.151832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30025cde-8cec-42e6-8bfd-f5f0ad71043e None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:48.184785 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445079cb-4afe-460a-848f-a8db966de897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:48.185419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445079cb-4afe-460a-848f-a8db966de897 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:48.185702 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445079cb-4afe-460a-848f-a8db966de897 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:48.185978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445079cb-4afe-460a-848f-a8db966de897 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:48.186631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445079cb-4afe-460a-848f-a8db966de897 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:48.188079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445079cb-4afe-460a-848f-a8db966de897 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:48.213811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 30/57] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Sep 28 08:23:48 2022] POST /identity/v3/projects => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:23:50.353360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16886c2-f66b-4345-ba8a-eb786ef3128e None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:50.354144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16886c2-f66b-4345-ba8a-eb786ef3128e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:50.354144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16886c2-f66b-4345-ba8a-eb786ef3128e None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:50.355272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 28/58] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:50.361411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c1567-6b0b-4e87-8e4d-951c80cf2453 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:50.361819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c1567-6b0b-4e87-8e4d-951c80cf2453 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:50.361965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c1567-6b0b-4e87-8e4d-951c80cf2453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:50.380540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-1e7c1567-6b0b-4e87-8e4d-951c80cf2453 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:50.381763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1e7c1567-6b0b-4e87-8e4d-951c80cf2453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:50.446121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 31/59] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:23:50.723309 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16886c2-f66b-4345-ba8a-eb786ef3128e None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:50.756902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:50.828410 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16886c2-f66b-4345-ba8a-eb786ef3128e None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:50.865646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662978b-c4bd-40f1-bb9b-8b6bf4d87004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:50.866889 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662978b-c4bd-40f1-bb9b-8b6bf4d87004 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:50.866889 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662978b-c4bd-40f1-bb9b-8b6bf4d87004 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:50.866889 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662978b-c4bd-40f1-bb9b-8b6bf4d87004 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:50.868151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662978b-c4bd-40f1-bb9b-8b6bf4d87004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:50.869709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662978b-c4bd-40f1-bb9b-8b6bf4d87004 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:50.871327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 29/60] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:23:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:50.877585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c1567-6b0b-4e87-8e4d-951c80cf2453 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:50.904630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:50.978308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c1567-6b0b-4e87-8e4d-951c80cf2453 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:51.011359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f52e5-00ab-4cb4-a54d-3c7dd3b5d463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:51.011970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f52e5-00ab-4cb4-a54d-3c7dd3b5d463 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:51.012257 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f52e5-00ab-4cb4-a54d-3c7dd3b5d463 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:51.012537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f52e5-00ab-4cb4-a54d-3c7dd3b5d463 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:51.013216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f52e5-00ab-4cb4-a54d-3c7dd3b5d463 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:51.014666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f52e5-00ab-4cb4-a54d-3c7dd3b5d463 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:51.041268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 32/61] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:23:50 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:23:54.475583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ae617-777e-4748-a0a6-9a8f3cb2483f None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:54.475583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ae617-777e-4748-a0a6-9a8f3cb2483f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:54.475583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ae617-777e-4748-a0a6-9a8f3cb2483f None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:54.475583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 30/62] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:54.481166 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900357e3-bb33-4739-aec1-27d69e17268f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:54.481565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900357e3-bb33-4739-aec1-27d69e17268f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:54.481729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900357e3-bb33-4739-aec1-27d69e17268f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:54.510703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-900357e3-bb33-4739-aec1-27d69e17268f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:54.512235 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-900357e3-bb33-4739-aec1-27d69e17268f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:54.610674 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 33/63] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:23:54.723702 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c457374-3f2d-4e3e-ab08-4e445720f4de None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:54.724153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c457374-3f2d-4e3e-ab08-4e445720f4de None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:54.724540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c457374-3f2d-4e3e-ab08-4e445720f4de None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:54.726314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 31/64] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:54.731891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcceafbd-da73-4187-9bb6-d77e390d1ed1 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:54.732391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcceafbd-da73-4187-9bb6-d77e390d1ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:54.732807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcceafbd-da73-4187-9bb6-d77e390d1ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:54.754096 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-bcceafbd-da73-4187-9bb6-d77e390d1ed1 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:54.755617 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-bcceafbd-da73-4187-9bb6-d77e390d1ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:54.760547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:54.786037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb030e-2155-4af8-9bab-e98cbc5cce73 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:54.786320 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb030e-2155-4af8-9bab-e98cbc5cce73 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:54.786797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb030e-2155-4af8-9bab-e98cbc5cce73 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:54.788278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 32/65] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:54.823763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6c7f8-114f-411b-8c8c-74a33484ec6f None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:54.824272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6c7f8-114f-411b-8c8c-74a33484ec6f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:54.824691 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6c7f8-114f-411b-8c8c-74a33484ec6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:54.852662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-a7a6c7f8-114f-411b-8c8c-74a33484ec6f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:54.854335 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a7a6c7f8-114f-411b-8c8c-74a33484ec6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:54.872437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 34/66] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:23:54.945116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcceafbd-da73-4187-9bb6-d77e390d1ed1 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:54.974232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 33/67] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:23:54.994490 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53c3f-0fe8-44b8-be10-1c98a985cb87 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:54.995108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53c3f-0fe8-44b8-be10-1c98a985cb87 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:54.995586 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53c3f-0fe8-44b8-be10-1c98a985cb87 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:54.997200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 34/68] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:23:54.998594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39958b-b6c6-4b80-a2aa-42424abcceea None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:54.999090 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39958b-b6c6-4b80-a2aa-42424abcceea None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:54.999591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39958b-b6c6-4b80-a2aa-42424abcceea None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:55.001050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 35/69] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:55.010437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368c25c-cb18-4da3-b49b-bf414973a8e3 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:55.011072 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368c25c-cb18-4da3-b49b-bf414973a8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:55.011538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368c25c-cb18-4da3-b49b-bf414973a8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:55.033005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-0368c25c-cb18-4da3-b49b-bf414973a8e3 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:55.035506 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-0368c25c-cb18-4da3-b49b-bf414973a8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:55.067475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:55.083083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcceafbd-da73-4187-9bb6-d77e390d1ed1 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:55.129582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0626048-2708-47ad-a7bc-c635326f4156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:55.130357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0626048-2708-47ad-a7bc-c635326f4156 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:55.130759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0626048-2708-47ad-a7bc-c635326f4156 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:55.131128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0626048-2708-47ad-a7bc-c635326f4156 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:55.133547 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 36/70] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:23:55.135550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761eee31-fc94-4d62-a4e1-db20f2df893a None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:55.136222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761eee31-fc94-4d62-a4e1-db20f2df893a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:55.136711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761eee31-fc94-4d62-a4e1-db20f2df893a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:55.150515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:55.154085 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0626048-2708-47ad-a7bc-c635326f4156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-c0626048-2708-47ad-a7bc-c635326f4156 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:23:55.161508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:23:55.162367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:23:55.163521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 35/71] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:23:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:23:55.168521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0626048-2708-47ad-a7bc-c635326f4156 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:55.177187 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-761eee31-fc94-4d62-a4e1-db20f2df893a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:55.178397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-761eee31-fc94-4d62-a4e1-db20f2df893a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:55.285987 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0626048-2708-47ad-a7bc-c635326f4156 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:55.303703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 37/72] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:23:55.304358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761eee31-fc94-4d62-a4e1-db20f2df893a None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:55.345431 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb5206-1c40-4660-ae36-0377488740c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:55.346380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb5206-1c40-4660-ae36-0377488740c8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:55.346812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb5206-1c40-4660-ae36-0377488740c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:55.347234 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb5206-1c40-4660-ae36-0377488740c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:55.348263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb5206-1c40-4660-ae36-0377488740c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:55.348739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb5206-1c40-4660-ae36-0377488740c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:55.351083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb5206-1c40-4660-ae36-0377488740c8 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:55.364101 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 36/73] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:23:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:55.365379 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb5206-1c40-4660-ae36-0377488740c8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:55.409816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:55.424805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761eee31-fc94-4d62-a4e1-db20f2df893a None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:55.441506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:55.461646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ba6fa-b0ea-4f55-b1e7-9bc0697e7682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:55.462282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ba6fa-b0ea-4f55-b1e7-9bc0697e7682 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:55.462561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ba6fa-b0ea-4f55-b1e7-9bc0697e7682 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:55.462831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ba6fa-b0ea-4f55-b1e7-9bc0697e7682 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:55.473486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ba6fa-b0ea-4f55-b1e7-9bc0697e7682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-ee3ba6fa-b0ea-4f55-b1e7-9bc0697e7682 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:23:55.478491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:23:55.479764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 38/74] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:23:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:23:55.480576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ba6fa-b0ea-4f55-b1e7-9bc0697e7682 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:55.492222 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb5206-1c40-4660-ae36-0377488740c8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:55.531370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be96d1b-2d95-47f6-b735-218cb257b942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:55.532002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be96d1b-2d95-47f6-b735-218cb257b942 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:55.532286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be96d1b-2d95-47f6-b735-218cb257b942 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:55.532559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be96d1b-2d95-47f6-b735-218cb257b942 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:55.543629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be96d1b-2d95-47f6-b735-218cb257b942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-1be96d1b-2d95-47f6-b735-218cb257b942 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:23:55.548446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:23:55.549343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:23:55.550095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 37/75] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:23:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:23:55.550729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be96d1b-2d95-47f6-b735-218cb257b942 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:55.598876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ba6fa-b0ea-4f55-b1e7-9bc0697e7682 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:55.634024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce332ae9-6141-4c14-b265-54ebe57f5ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:55.634666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce332ae9-6141-4c14-b265-54ebe57f5ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:55.634951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce332ae9-6141-4c14-b265-54ebe57f5ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:55.635224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce332ae9-6141-4c14-b265-54ebe57f5ca2 None admin] PATH_INFO: `/v3/users/0ff656a811aa4e698617e13e80ad9d73` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:55.649174 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce332ae9-6141-4c14-b265-54ebe57f5ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=0ff656a811aa4e698617e13e80ad9d73)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:55.650908 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce332ae9-6141-4c14-b265-54ebe57f5ca2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:55.652602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 39/76] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:23:55 2022] GET /identity/v3/users/0ff656a811aa4e698617e13e80ad9d73 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:55.652602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce332ae9-6141-4c14-b265-54ebe57f5ca2 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:55.662444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:55.671540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be96d1b-2d95-47f6-b735-218cb257b942 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:55.695376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:55.707381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82495e9e-c63b-4e0f-b9a0-9a7d900d7269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:55.708013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82495e9e-c63b-4e0f-b9a0-9a7d900d7269 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:55.708295 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82495e9e-c63b-4e0f-b9a0-9a7d900d7269 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:55.708575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82495e9e-c63b-4e0f-b9a0-9a7d900d7269 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:55.709722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82495e9e-c63b-4e0f-b9a0-9a7d900d7269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:55.711151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82495e9e-c63b-4e0f-b9a0-9a7d900d7269 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:55.712924 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 38/77] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:23:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:55.713638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82495e9e-c63b-4e0f-b9a0-9a7d900d7269 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:55.758758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce332ae9-6141-4c14-b265-54ebe57f5ca2 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:55.792330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35457bb7-0468-4346-8d71-1521160633fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:55.792937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35457bb7-0468-4346-8d71-1521160633fd None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:55.793253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35457bb7-0468-4346-8d71-1521160633fd None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:55.793538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35457bb7-0468-4346-8d71-1521160633fd None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:55.794266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35457bb7-0468-4346-8d71-1521160633fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:55.794518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35457bb7-0468-4346-8d71-1521160633fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:55.795916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35457bb7-0468-4346-8d71-1521160633fd None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:55.804841 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 40/78] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:23:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:55.804841 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35457bb7-0468-4346-8d71-1521160633fd None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:55.819195 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82495e9e-c63b-4e0f-b9a0-9a7d900d7269 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:55.853708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f141ba-f91b-4642-9964-721d1c44a536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:55.854225 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f141ba-f91b-4642-9964-721d1c44a536 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:55.854633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f141ba-f91b-4642-9964-721d1c44a536 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:55.854836 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f141ba-f91b-4642-9964-721d1c44a536 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:55.855485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f141ba-f91b-4642-9964-721d1c44a536 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:55.855791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f141ba-f91b-4642-9964-721d1c44a536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:55.857498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f141ba-f91b-4642-9964-721d1c44a536 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:55.865954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 39/79] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:23:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:55.866824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f141ba-f91b-4642-9964-721d1c44a536 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:55.876518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:55.910613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35457bb7-0468-4346-8d71-1521160633fd None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:55.918747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:55.944528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a0e74-98b5-4a27-a1e7-ca0da48aae8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:55.945186 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a0e74-98b5-4a27-a1e7-ca0da48aae8a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:55.945475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a0e74-98b5-4a27-a1e7-ca0da48aae8a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:55.945763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a0e74-98b5-4a27-a1e7-ca0da48aae8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:55.955805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a0e74-98b5-4a27-a1e7-ca0da48aae8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-bd3a0e74-98b5-4a27-a1e7-ca0da48aae8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:23:55.961173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:23:55.962040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:23:55.962818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 41/80] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:23:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:23:55.963435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a0e74-98b5-4a27-a1e7-ca0da48aae8a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:55.969925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f141ba-f91b-4642-9964-721d1c44a536 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:56.005377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eaf4c5-443c-4a11-bb57-954e6d3f4eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:56.005990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eaf4c5-443c-4a11-bb57-954e6d3f4eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:56.006276 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eaf4c5-443c-4a11-bb57-954e6d3f4eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:56.006558 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eaf4c5-443c-4a11-bb57-954e6d3f4eb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:56.065744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a0e74-98b5-4a27-a1e7-ca0da48aae8a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:56.076825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eaf4c5-443c-4a11-bb57-954e6d3f4eb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:56.078460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eaf4c5-443c-4a11-bb57-954e6d3f4eb9 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:56.091043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:56.100085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d17327-477a-4517-b3d6-93f44f20a32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:56.100770 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17327-477a-4517-b3d6-93f44f20a32e None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:56.101053 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17327-477a-4517-b3d6-93f44f20a32e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:56.101359 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17327-477a-4517-b3d6-93f44f20a32e None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:56.102015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d17327-477a-4517-b3d6-93f44f20a32e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:56.103483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d17327-477a-4517-b3d6-93f44f20a32e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:56.130550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:56.132434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 42/81] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:23:55 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:23:56.133332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d17327-477a-4517-b3d6-93f44f20a32e None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:56.187977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 40/82] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:23:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:23:56.189790 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eaf4c5-443c-4a11-bb57-954e6d3f4eb9 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:56.262550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d17327-477a-4517-b3d6-93f44f20a32e None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:56.302434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d374dff-5f2a-41ed-9270-09fa6aa2233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:56.303096 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d374dff-5f2a-41ed-9270-09fa6aa2233b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:56.303379 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d374dff-5f2a-41ed-9270-09fa6aa2233b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:56.303874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d374dff-5f2a-41ed-9270-09fa6aa2233b None admin] PATH_INFO: `/v3/users/0ff656a811aa4e698617e13e80ad9d73` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:56.304902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d374dff-5f2a-41ed-9270-09fa6aa2233b None admin] RBAC: Authorizing `identity:get_user(user_id=0ff656a811aa4e698617e13e80ad9d73)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:56.306358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d374dff-5f2a-41ed-9270-09fa6aa2233b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:56.306799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eaf4c5-443c-4a11-bb57-954e6d3f4eb9 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:56.309959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 43/83] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:23:56 2022] GET /identity/v3/users/0ff656a811aa4e698617e13e80ad9d73 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:56.309959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d374dff-5f2a-41ed-9270-09fa6aa2233b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:56.353905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97562a2-a92d-42e2-8c23-ba181be6a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:56.354692 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97562a2-a92d-42e2-8c23-ba181be6a915 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:56.354816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97562a2-a92d-42e2-8c23-ba181be6a915 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:56.355066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97562a2-a92d-42e2-8c23-ba181be6a915 None admin] PATH_INFO: `/v3/users/3300b0df5e5741c8b0d940d0202bb604` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:56.356292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97562a2-a92d-42e2-8c23-ba181be6a915 None admin] RBAC: Authorizing `identity:get_user(user_id=3300b0df5e5741c8b0d940d0202bb604)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:56.357849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97562a2-a92d-42e2-8c23-ba181be6a915 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:56.360582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 41/84] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:23:56 2022] GET /identity/v3/users/3300b0df5e5741c8b0d940d0202bb604 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:56.360582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97562a2-a92d-42e2-8c23-ba181be6a915 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:56.381202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:56.391499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:56.448834 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d374dff-5f2a-41ed-9270-09fa6aa2233b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:56.491408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01703744-9946-4213-8832-2a4a0f62af7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:56.492800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01703744-9946-4213-8832-2a4a0f62af7a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:56.493148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01703744-9946-4213-8832-2a4a0f62af7a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:56.493402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01703744-9946-4213-8832-2a4a0f62af7a None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:56.495239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01703744-9946-4213-8832-2a4a0f62af7a None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:56.495239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01703744-9946-4213-8832-2a4a0f62af7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:56.497020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01703744-9946-4213-8832-2a4a0f62af7a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:56.499744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97562a2-a92d-42e2-8c23-ba181be6a915 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:56.508246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 44/85] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:23:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:56.509574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01703744-9946-4213-8832-2a4a0f62af7a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:56.549975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db050c0-b80b-4cea-a485-a6bccacd625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:56.550628 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db050c0-b80b-4cea-a485-a6bccacd625f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:56.550912 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db050c0-b80b-4cea-a485-a6bccacd625f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:56.551188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db050c0-b80b-4cea-a485-a6bccacd625f None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:56.554003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db050c0-b80b-4cea-a485-a6bccacd625f None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:56.556444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db050c0-b80b-4cea-a485-a6bccacd625f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:56.558739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 42/86] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:23:56 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:56.559862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db050c0-b80b-4cea-a485-a6bccacd625f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:56.637145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01703744-9946-4213-8832-2a4a0f62af7a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:56.652531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:56.676018 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c31716-2d28-4aef-9e3f-40cddfbbf5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:56.676642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c31716-2d28-4aef-9e3f-40cddfbbf5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:56.676979 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c31716-2d28-4aef-9e3f-40cddfbbf5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:56.677299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c31716-2d28-4aef-9e3f-40cddfbbf5e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:56.694472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db050c0-b80b-4cea-a485-a6bccacd625f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:56.706294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:56.735425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c60c77-880a-46fd-9f2d-714d13d0d2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:56.736049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c60c77-880a-46fd-9f2d-714d13d0d2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:56.737004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c60c77-880a-46fd-9f2d-714d13d0d2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:56.737220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c60c77-880a-46fd-9f2d-714d13d0d2a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:56.771910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c31716-2d28-4aef-9e3f-40cddfbbf5e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:56.774685 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c31716-2d28-4aef-9e3f-40cddfbbf5e6 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:56.823930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c60c77-880a-46fd-9f2d-714d13d0d2a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:56.825839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c60c77-880a-46fd-9f2d-714d13d0d2a9 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:56.919369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 45/87] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:23:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:23:56.920256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c31716-2d28-4aef-9e3f-40cddfbbf5e6 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:56.936753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:56.964929 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:56.986522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 43/88] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:23:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:23:56.988008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c60c77-880a-46fd-9f2d-714d13d0d2a9 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:57.036084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c31716-2d28-4aef-9e3f-40cddfbbf5e6 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:57.073827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf581a-a4f1-4098-9e38-5d35f23a0fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:57.074517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf581a-a4f1-4098-9e38-5d35f23a0fdb None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:57.074798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf581a-a4f1-4098-9e38-5d35f23a0fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:57.075070 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf581a-a4f1-4098-9e38-5d35f23a0fdb None admin] PATH_INFO: `/v3/users/0ff656a811aa4e698617e13e80ad9d73` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:57.076227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf581a-a4f1-4098-9e38-5d35f23a0fdb None admin] RBAC: Authorizing `identity:get_user(user_id=0ff656a811aa4e698617e13e80ad9d73)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:57.077941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf581a-a4f1-4098-9e38-5d35f23a0fdb None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:57.084824 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 46/89] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:23:56 2022] GET /identity/v3/users/0ff656a811aa4e698617e13e80ad9d73 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:57.085772 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf581a-a4f1-4098-9e38-5d35f23a0fdb None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:57.164918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c60c77-880a-46fd-9f2d-714d13d0d2a9 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:57.184262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:57.208202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf581a-a4f1-4098-9e38-5d35f23a0fdb None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:57.238326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617b10b-663e-4be0-b2c7-78b99f0c756b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:57.239152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617b10b-663e-4be0-b2c7-78b99f0c756b None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:57.239307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617b10b-663e-4be0-b2c7-78b99f0c756b None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:57.239647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617b10b-663e-4be0-b2c7-78b99f0c756b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:57.240448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617b10b-663e-4be0-b2c7-78b99f0c756b None admin] RBAC: Adding query filter params (role.id=4c80373ee4c34171a7f29e5ab6c09ce0, scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859, user.id=0ff656a811aa4e698617e13e80ad9d73) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:57.240824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617b10b-663e-4be0-b2c7-78b99f0c756b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:57.242364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617b10b-663e-4be0-b2c7-78b99f0c756b None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:57.249036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 44/90] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:23:56 2022] GET /identity/v3/role_assignments?user.id=0ff656a811aa4e698617e13e80ad9d73&scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859&role.id=4c80373ee4c34171a7f29e5ab6c09ce0 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:57.250087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617b10b-663e-4be0-b2c7-78b99f0c756b None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:57.260466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd3a96-cbef-4bec-8276-cb4590837586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:57.261962 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd3a96-cbef-4bec-8276-cb4590837586 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:57.262602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd3a96-cbef-4bec-8276-cb4590837586 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:57.264020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd3a96-cbef-4bec-8276-cb4590837586 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:57.267404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd3a96-cbef-4bec-8276-cb4590837586 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:57.272093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd3a96-cbef-4bec-8276-cb4590837586 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:57.275189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 47/91] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:23:57 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:57.276181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd3a96-cbef-4bec-8276-cb4590837586 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:57.319486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:57.387173 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617b10b-663e-4be0-b2c7-78b99f0c756b None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:57.405678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd3a96-cbef-4bec-8276-cb4590837586 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:57.435835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39baadc-1f4f-4c93-8b97-790c84d94f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:57.437555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39baadc-1f4f-4c93-8b97-790c84d94f10 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:57.438067 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39baadc-1f4f-4c93-8b97-790c84d94f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:57.438563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39baadc-1f4f-4c93-8b97-790c84d94f10 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:57.441289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39baadc-1f4f-4c93-8b97-790c84d94f10 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:57.443953 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39baadc-1f4f-4c93-8b97-790c84d94f10 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:57.448751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 45/92] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:23:57 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:57.450488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39baadc-1f4f-4c93-8b97-790c84d94f10 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:57.454752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecd6e9-5740-4d29-ba19-d6977a048506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:57.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecd6e9-5740-4d29-ba19-d6977a048506 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:57.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecd6e9-5740-4d29-ba19-d6977a048506 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:57.456328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecd6e9-5740-4d29-ba19-d6977a048506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:57.477842 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:57.549615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecd6e9-5740-4d29-ba19-d6977a048506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:57.552468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecd6e9-5740-4d29-ba19-d6977a048506 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:57.586653 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39baadc-1f4f-4c93-8b97-790c84d94f10 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:57.613303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:57.657811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80382b01-029b-4ad6-bb15-0a4d7e526744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:57.658971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80382b01-029b-4ad6-bb15-0a4d7e526744 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:57.659253 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80382b01-029b-4ad6-bb15-0a4d7e526744 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:57.659526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80382b01-029b-4ad6-bb15-0a4d7e526744 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:57.660261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80382b01-029b-4ad6-bb15-0a4d7e526744 None admin] RBAC: Adding query filter params (role.id=89694ebde7824b8190bd5b15ece2b11e, scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859, user.id=0ff656a811aa4e698617e13e80ad9d73) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:57.660576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80382b01-029b-4ad6-bb15-0a4d7e526744 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:57.662124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80382b01-029b-4ad6-bb15-0a4d7e526744 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:57.669751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 46/93] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:23:57 2022] GET /identity/v3/role_assignments?user.id=0ff656a811aa4e698617e13e80ad9d73&scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859&role.id=89694ebde7824b8190bd5b15ece2b11e => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:57.671017 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80382b01-029b-4ad6-bb15-0a4d7e526744 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:57.719535 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 48/94] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:23:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:23:57.722991 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecd6e9-5740-4d29-ba19-d6977a048506 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:57.829126 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:57.882965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80382b01-029b-4ad6-bb15-0a4d7e526744 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:57.928564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95149d8-dedc-4edd-a69c-f86633846642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:57.930105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95149d8-dedc-4edd-a69c-f86633846642 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:57.933221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95149d8-dedc-4edd-a69c-f86633846642 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:57.934551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95149d8-dedc-4edd-a69c-f86633846642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:57.935980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95149d8-dedc-4edd-a69c-f86633846642 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859, user.id=3300b0df5e5741c8b0d940d0202bb604) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:57.936703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95149d8-dedc-4edd-a69c-f86633846642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:57.939009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95149d8-dedc-4edd-a69c-f86633846642 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:57.942082 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecd6e9-5740-4d29-ba19-d6977a048506 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:57.954272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 47/95] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:23:57 2022] GET /identity/v3/role_assignments?user.id=3300b0df5e5741c8b0d940d0202bb604&scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:58.022138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3cc6f-4a3b-4e54-a3ca-77b74125547c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:58.022711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3cc6f-4a3b-4e54-a3ca-77b74125547c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:58.022995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3cc6f-4a3b-4e54-a3ca-77b74125547c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:58.023276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3cc6f-4a3b-4e54-a3ca-77b74125547c None admin] PATH_INFO: `/v3/projects/bc3122d8dc20463bb81537175e4c5d28` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:58.027087 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3cc6f-4a3b-4e54-a3ca-77b74125547c None admin] RBAC: Authorizing `identity:get_project(project_id=bc3122d8dc20463bb81537175e4c5d28)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:58.028594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3cc6f-4a3b-4e54-a3ca-77b74125547c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:58.030405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 49/96] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:23:57 2022] GET /identity/v3/projects/bc3122d8dc20463bb81537175e4c5d28 => generated 312 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:23:58.036923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95149d8-dedc-4edd-a69c-f86633846642 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:58.140237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:58.229605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95149d8-dedc-4edd-a69c-f86633846642 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:58.282230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429a294-834f-4de8-b29b-3b6f5d982b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:58.283971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429a294-834f-4de8-b29b-3b6f5d982b49 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:58.284615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429a294-834f-4de8-b29b-3b6f5d982b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:58.285280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429a294-834f-4de8-b29b-3b6f5d982b49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:58.286629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429a294-834f-4de8-b29b-3b6f5d982b49 None admin] RBAC: Adding query filter params (role.id=4c80373ee4c34171a7f29e5ab6c09ce0, scope.project.id=bc3122d8dc20463bb81537175e4c5d28, user.id=0ff656a811aa4e698617e13e80ad9d73) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:23:58.287444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429a294-834f-4de8-b29b-3b6f5d982b49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:58.290078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429a294-834f-4de8-b29b-3b6f5d982b49 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:58.299831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 48/97] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:23:58 2022] GET /identity/v3/role_assignments?user.id=0ff656a811aa4e698617e13e80ad9d73&scope.project.id=bc3122d8dc20463bb81537175e4c5d28&role.id=4c80373ee4c34171a7f29e5ab6c09ce0 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:58.704198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86990120-cf7b-4b75-973b-14b9ae719aaa None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:58.704689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86990120-cf7b-4b75-973b-14b9ae719aaa None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:58.704943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86990120-cf7b-4b75-973b-14b9ae719aaa None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:58.707360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 50/98] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:23:58.713319 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c69ee-70e4-41ca-bcd4-42893559f556 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:58.713889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c69ee-70e4-41ca-bcd4-42893559f556 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:58.714327 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c69ee-70e4-41ca-bcd4-42893559f556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:58.744142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-195c69ee-70e4-41ca-bcd4-42893559f556 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:58.745382 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-195c69ee-70e4-41ca-bcd4-42893559f556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:58.832698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 49/99] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:23:59.183271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ceff86-46dc-4b60-9cdd-7c3b7510f5a2 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:59.183607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ceff86-46dc-4b60-9cdd-7c3b7510f5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:59.183782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ceff86-46dc-4b60-9cdd-7c3b7510f5a2 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:59.185106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 51/100] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:23:59.191119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a5a76-f3b6-45e0-aed4-9847ce2bc721 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:59.191407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a5a76-f3b6-45e0-aed4-9847ce2bc721 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:59.191683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a5a76-f3b6-45e0-aed4-9847ce2bc721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:59.219985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-6e0a5a76-f3b6-45e0-aed4-9847ce2bc721 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:23:59.221286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-6e0a5a76-f3b6-45e0-aed4-9847ce2bc721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:23:59.263530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:59.307509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 50/101] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:23:59.418979 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ceff86-46dc-4b60-9cdd-7c3b7510f5a2 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:59.577796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ceff86-46dc-4b60-9cdd-7c3b7510f5a2 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:59.602050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:23:59.628362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47cb07-67b1-4512-97d7-b5b7e2be6853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:59.629010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47cb07-67b1-4512-97d7-b5b7e2be6853 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:59.629324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47cb07-67b1-4512-97d7-b5b7e2be6853 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:59.629596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47cb07-67b1-4512-97d7-b5b7e2be6853 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:59.643991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a5a76-f3b6-45e0-aed4-9847ce2bc721 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:59.646981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47cb07-67b1-4512-97d7-b5b7e2be6853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-bc47cb07-67b1-4512-97d7-b5b7e2be6853 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:23:59.652330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:23:59.653390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:23:59.654317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:23:59.654317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:23:59.654317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:23:59.654317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:23:59.654317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:23:59.654317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:23:59.654317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:23:59.654317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:23:59.654317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:23:59.654317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:23:59.654317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:23:59.654317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:23:59.656218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 52/102] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:23:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:23:59.664044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47cb07-67b1-4512-97d7-b5b7e2be6853 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:59.823972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a5a76-f3b6-45e0-aed4-9847ce2bc721 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:59.895236 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af935c-6b49-44c3-b882-d3e265130f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:23:59.895927 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af935c-6b49-44c3-b882-d3e265130f7a None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:23:59.896208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af935c-6b49-44c3-b882-d3e265130f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:23:59.896486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af935c-6b49-44c3-b882-d3e265130f7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:23:59.901968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af935c-6b49-44c3-b882-d3e265130f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:23:59.903673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af935c-6b49-44c3-b882-d3e265130f7a None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:23:59.905502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 51/103] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:23:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:23:59.906284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47cb07-67b1-4512-97d7-b5b7e2be6853 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:23:59.912112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af935c-6b49-44c3-b882-d3e265130f7a None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:23:59.993515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:00.003537 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e33bac-7272-4444-ac17-d0c1bc0c3d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:00.004606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e33bac-7272-4444-ac17-d0c1bc0c3d62 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:00.005014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e33bac-7272-4444-ac17-d0c1bc0c3d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:00.005487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e33bac-7272-4444-ac17-d0c1bc0c3d62 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:00.006481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e33bac-7272-4444-ac17-d0c1bc0c3d62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:00.006929 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e33bac-7272-4444-ac17-d0c1bc0c3d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:00.009011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e33bac-7272-4444-ac17-d0c1bc0c3d62 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:00.052335 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 53/104] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:23:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:00.062497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e33bac-7272-4444-ac17-d0c1bc0c3d62 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:00.090372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af935c-6b49-44c3-b882-d3e265130f7a None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:00.181288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:00.202052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a5bba-0678-4b81-8010-f4837f619774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:00.202952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a5bba-0678-4b81-8010-f4837f619774 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:00.203371 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a5bba-0678-4b81-8010-f4837f619774 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:00.203951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a5bba-0678-4b81-8010-f4837f619774 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:00.205189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a5bba-0678-4b81-8010-f4837f619774 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:00.207513 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a5bba-0678-4b81-8010-f4837f619774 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:00.279202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 52/105] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:23:59 2022] POST /identity/v3/projects => generated 302 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:24:00.297470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e33bac-7272-4444-ac17-d0c1bc0c3d62 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:00.355961 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a3cc3-93ce-45b6-9cf0-684ab1c1af9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:00.356983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a3cc3-93ce-45b6-9cf0-684ab1c1af9a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:00.357502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a3cc3-93ce-45b6-9cf0-684ab1c1af9a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:00.357941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a3cc3-93ce-45b6-9cf0-684ab1c1af9a None admin] PATH_INFO: `/v3/users/a06a548fe9a74214ba8da3f9ece2d6fb` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:00.379435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a3cc3-93ce-45b6-9cf0-684ab1c1af9a None admin] RBAC: Authorizing `identity:get_user(user_id=a06a548fe9a74214ba8da3f9ece2d6fb)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:00.381177 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a3cc3-93ce-45b6-9cf0-684ab1c1af9a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:00.383051 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 54/106] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:00 2022] GET /identity/v3/users/a06a548fe9a74214ba8da3f9ece2d6fb => generated 289 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:00.390552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a5bba-0678-4b81-8010-f4837f619774 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:00.564833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:00.575925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a5bba-0678-4b81-8010-f4837f619774 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:00.621000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664c131-5b4a-4858-bf74-5780fbbdb8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:00.621644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664c131-5b4a-4858-bf74-5780fbbdb8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:00.621931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664c131-5b4a-4858-bf74-5780fbbdb8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:00.622207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664c131-5b4a-4858-bf74-5780fbbdb8a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:00.740732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664c131-5b4a-4858-bf74-5780fbbdb8a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:00.742237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664c131-5b4a-4858-bf74-5780fbbdb8a6 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:00.865588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:00.869735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 53/107] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:00.875290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a3cc3-93ce-45b6-9cf0-684ab1c1af9a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:00.995221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:01.002805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a3cc3-93ce-45b6-9cf0-684ab1c1af9a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:01.042767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3284d38-8f38-4353-a65b-aba27cbc6f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:01.043738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3284d38-8f38-4353-a65b-aba27cbc6f05 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:01.044266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3284d38-8f38-4353-a65b-aba27cbc6f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:01.044751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3284d38-8f38-4353-a65b-aba27cbc6f05 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:01.046190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3284d38-8f38-4353-a65b-aba27cbc6f05 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:01.047825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3284d38-8f38-4353-a65b-aba27cbc6f05 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:01.056480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664c131-5b4a-4858-bf74-5780fbbdb8a6 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:01.059887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 55/108] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:00 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:01.176853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664c131-5b4a-4858-bf74-5780fbbdb8a6 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:01.222396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53640197-902c-4f59-8f9a-5b0fb27494a6 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:01.222958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53640197-902c-4f59-8f9a-5b0fb27494a6 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:01.223405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53640197-902c-4f59-8f9a-5b0fb27494a6 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:01.225032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 56/109] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:24:01.227336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9927d2-8a73-403d-8caf-4174be4440b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:01.228157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9927d2-8a73-403d-8caf-4174be4440b0 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:01.228612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9927d2-8a73-403d-8caf-4174be4440b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:01.229063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9927d2-8a73-403d-8caf-4174be4440b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:01.230768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e8629-79f9-4232-bdac-0397b0d4fa6f None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:01.231225 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e8629-79f9-4232-bdac-0397b0d4fa6f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:01.231668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e8629-79f9-4232-bdac-0397b0d4fa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:01.239187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9927d2-8a73-403d-8caf-4174be4440b0 None admin] RBAC: Adding query filter params (role.id=718dd530c67f4a1891e292e729330f51, scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859, user.id=a06a548fe9a74214ba8da3f9ece2d6fb) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:01.240006 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9927d2-8a73-403d-8caf-4174be4440b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:01.242608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9927d2-8a73-403d-8caf-4174be4440b0 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:01.256298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 54/110] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:01 2022] GET /identity/v3/role_assignments?user.id=a06a548fe9a74214ba8da3f9ece2d6fb&scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859&role.id=718dd530c67f4a1891e292e729330f51 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:01.258133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-cb6e8629-79f9-4232-bdac-0397b0d4fa6f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:01.259649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-cb6e8629-79f9-4232-bdac-0397b0d4fa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:01.382830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 57/111] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:01.608394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e564a04-3756-424f-88eb-877a86259d4a None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:01.608818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e564a04-3756-424f-88eb-877a86259d4a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:01.609238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e564a04-3756-424f-88eb-877a86259d4a None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:01.611949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 55/112] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:01.617458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807f412-ee95-4810-9da8-32b3f45c6422 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:01.618023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807f412-ee95-4810-9da8-32b3f45c6422 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:01.618502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807f412-ee95-4810-9da8-32b3f45c6422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:01.645947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-6807f412-ee95-4810-9da8-32b3f45c6422 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:01.647506 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6807f412-ee95-4810-9da8-32b3f45c6422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:01.685520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da827d-c71f-431d-915b-99285c2c9e49 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:01.685906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da827d-c71f-431d-915b-99285c2c9e49 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:01.686294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da827d-c71f-431d-915b-99285c2c9e49 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:01.688315 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 56/113] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:01.698085 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de1c27-43d8-4fa2-bfa0-d8662d0b1a77 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:01.698490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de1c27-43d8-4fa2-bfa0-d8662d0b1a77 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:01.698881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de1c27-43d8-4fa2-bfa0-d8662d0b1a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:01.743911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-76de1c27-43d8-4fa2-bfa0-d8662d0b1a77 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:01.745169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-76de1c27-43d8-4fa2-bfa0-d8662d0b1a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:01.762654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 58/114] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:01.841548 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:01.880133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 57/115] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:01.935059 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807f412-ee95-4810-9da8-32b3f45c6422 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:01.971185 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:02.061950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807f412-ee95-4810-9da8-32b3f45c6422 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:02.079759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5591b4-8a0f-4cc8-9b18-311f9dd701b9 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:02.080358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5591b4-8a0f-4cc8-9b18-311f9dd701b9 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:02.080652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5591b4-8a0f-4cc8-9b18-311f9dd701b9 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:02.082536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 58/116] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:02.088178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842f120-955d-445f-8ecd-5d61275ac2bc None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:02.088559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842f120-955d-445f-8ecd-5d61275ac2bc None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:02.088854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842f120-955d-445f-8ecd-5d61275ac2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:02.107919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacaaf6-3103-472d-98a7-cd1f9078e9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:02.108496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacaaf6-3103-472d-98a7-cd1f9078e9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:02.108781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacaaf6-3103-472d-98a7-cd1f9078e9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:02.109057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacaaf6-3103-472d-98a7-cd1f9078e9b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:02.126358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-7842f120-955d-445f-8ecd-5d61275ac2bc None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:02.127279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebacaaf6-3103-472d-98a7-cd1f9078e9b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:02.127642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7842f120-955d-445f-8ecd-5d61275ac2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-ebacaaf6-3103-472d-98a7-cd1f9078e9b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:02.132284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:02.133205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:02.134137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 59/117] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:24:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:02.139338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacaaf6-3103-472d-98a7-cd1f9078e9b7 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:02.248393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 59/118] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:02.252975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842f120-955d-445f-8ecd-5d61275ac2bc None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:02.327599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:02.354135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacaaf6-3103-472d-98a7-cd1f9078e9b7 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:02.370212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842f120-955d-445f-8ecd-5d61275ac2bc None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:02.385277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:02.419753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de3a35-7cb9-4e92-ac60-c368c6a03ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:02.420725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de3a35-7cb9-4e92-ac60-c368c6a03ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:02.421176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de3a35-7cb9-4e92-ac60-c368c6a03ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:02.421655 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de3a35-7cb9-4e92-ac60-c368c6a03ee9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:02.422952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7e5ab-a21d-4657-9ccd-5d1d7c5ab842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:02.423718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7e5ab-a21d-4657-9ccd-5d1d7c5ab842 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:02.424087 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7e5ab-a21d-4657-9ccd-5d1d7c5ab842 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:02.424468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7e5ab-a21d-4657-9ccd-5d1d7c5ab842 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:02.425291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7e5ab-a21d-4657-9ccd-5d1d7c5ab842 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:02.425683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7e5ab-a21d-4657-9ccd-5d1d7c5ab842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:02.427161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7e5ab-a21d-4657-9ccd-5d1d7c5ab842 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:02.443325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de3a35-7cb9-4e92-ac60-c368c6a03ee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:02.449997 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 60/119] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:02.456657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7e5ab-a21d-4657-9ccd-5d1d7c5ab842 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-68de3a35-7cb9-4e92-ac60-c368c6a03ee9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:02.456922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:02.457931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:02.458886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:02.458886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:02.458886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:02.458886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:02.458886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:02.458886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:02.458886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:02.458886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:02.458886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:02.458886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:02.458886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:02.458886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:02.460936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 60/120] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:02.472693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de3a35-7cb9-4e92-ac60-c368c6a03ee9 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:02.644302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de3a35-7cb9-4e92-ac60-c368c6a03ee9 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:02.667836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7e5ab-a21d-4657-9ccd-5d1d7c5ab842 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:02.680336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f52b6-900f-4e51-8bc5-d1cfd03e0eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:02.681293 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f52b6-900f-4e51-8bc5-d1cfd03e0eee None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:02.681780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f52b6-900f-4e51-8bc5-d1cfd03e0eee None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:02.682283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f52b6-900f-4e51-8bc5-d1cfd03e0eee None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:02.683158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f52b6-900f-4e51-8bc5-d1cfd03e0eee None admin] RBAC: Adding query filter params (name=member) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:02.683664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f52b6-900f-4e51-8bc5-d1cfd03e0eee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:02.685283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f52b6-900f-4e51-8bc5-d1cfd03e0eee None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:02.694736 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 61/121] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:02.695842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f52b6-900f-4e51-8bc5-d1cfd03e0eee None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:02.714135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684addc-e9cd-4ed7-afb3-22100a2948e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:02.714849 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684addc-e9cd-4ed7-afb3-22100a2948e9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:02.715161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684addc-e9cd-4ed7-afb3-22100a2948e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:02.715691 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684addc-e9cd-4ed7-afb3-22100a2948e9 None admin] PATH_INFO: `/v3/users/3300b0df5e5741c8b0d940d0202bb604` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:02.716857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684addc-e9cd-4ed7-afb3-22100a2948e9 None admin] RBAC: Authorizing `identity:get_user(user_id=3300b0df5e5741c8b0d940d0202bb604)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:02.719127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684addc-e9cd-4ed7-afb3-22100a2948e9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:02.725345 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 61/122] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:02 2022] GET /identity/v3/users/3300b0df5e5741c8b0d940d0202bb604 => generated 247 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:02.725345 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684addc-e9cd-4ed7-afb3-22100a2948e9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:02.754679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:02.767578 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:02.849434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684addc-e9cd-4ed7-afb3-22100a2948e9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:02.849636 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f52b6-900f-4e51-8bc5-d1cfd03e0eee None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:02.888315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a2b6b-10e3-48e7-90b4-cfe4ad1bc4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:02.889344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a2b6b-10e3-48e7-90b4-cfe4ad1bc4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:02.889939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a2b6b-10e3-48e7-90b4-cfe4ad1bc4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:02.890520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a2b6b-10e3-48e7-90b4-cfe4ad1bc4f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:02.893662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad599d3-c2ca-463c-b047-787591970df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:02.894409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad599d3-c2ca-463c-b047-787591970df1 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:02.894650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad599d3-c2ca-463c-b047-787591970df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:02.894913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad599d3-c2ca-463c-b047-787591970df1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:02.905168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a2b6b-10e3-48e7-90b4-cfe4ad1bc4f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:02.907470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad599d3-c2ca-463c-b047-787591970df1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-dad599d3-c2ca-463c-b047-787591970df1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:02.912872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:02.913956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:02.914708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 62/123] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:24:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:02.915342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad599d3-c2ca-463c-b047-787591970df1 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-596a2b6b-10e3-48e7-90b4-cfe4ad1bc4f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:02.921308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:02.922182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:02.922944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:02.922944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:02.922944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:02.922944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:02.922944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:02.922944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:02.922944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:02.922944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:02.922944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:02.922944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:02.922944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:02.922944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:02.925873 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 62/124] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:02.927066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a2b6b-10e3-48e7-90b4-cfe4ad1bc4f6 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:03.017518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:03.039940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:03.046582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad599d3-c2ca-463c-b047-787591970df1 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:03.049515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a2b6b-10e3-48e7-90b4-cfe4ad1bc4f6 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:03.089729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0b49d-d9e9-423e-8aa0-7b949e7828bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:03.090369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0b49d-d9e9-423e-8aa0-7b949e7828bc None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:03.090930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0b49d-d9e9-423e-8aa0-7b949e7828bc None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:03.091196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0b49d-d9e9-423e-8aa0-7b949e7828bc None admin] PATH_INFO: `/v3/users/0ff656a811aa4e698617e13e80ad9d73` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:03.092621 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0b49d-d9e9-423e-8aa0-7b949e7828bc None admin] RBAC: Authorizing `identity:get_user(user_id=0ff656a811aa4e698617e13e80ad9d73)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:03.094022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4393a7-5a12-43c8-b607-51fa926d5592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:03.094488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0b49d-d9e9-423e-8aa0-7b949e7828bc None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:03.094796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4393a7-5a12-43c8-b607-51fa926d5592 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:03.095150 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4393a7-5a12-43c8-b607-51fa926d5592 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:03.095522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4393a7-5a12-43c8-b607-51fa926d5592 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:03.095874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 63/125] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:02 2022] GET /identity/v3/users/0ff656a811aa4e698617e13e80ad9d73 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:03.097695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4393a7-5a12-43c8-b607-51fa926d5592 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:03.099172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4393a7-5a12-43c8-b607-51fa926d5592 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:03.100855 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 63/126] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:02 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:03.101856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0b49d-d9e9-423e-8aa0-7b949e7828bc None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:03.105978 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4393a7-5a12-43c8-b607-51fa926d5592 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:03.220386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4393a7-5a12-43c8-b607-51fa926d5592 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:03.229464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0b49d-d9e9-423e-8aa0-7b949e7828bc None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:03.259115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c417e0-7990-490a-b941-20ef021ec010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:03.259767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c417e0-7990-490a-b941-20ef021ec010 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:03.260056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c417e0-7990-490a-b941-20ef021ec010 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:03.260337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c417e0-7990-490a-b941-20ef021ec010 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:03.261117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c417e0-7990-490a-b941-20ef021ec010 None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:03.261462 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c417e0-7990-490a-b941-20ef021ec010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:03.262966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c417e0-7990-490a-b941-20ef021ec010 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:03.271646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1781b80-a678-4abd-a743-3d2eb357ca34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:03.272260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1781b80-a678-4abd-a743-3d2eb357ca34 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:03.272489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 64/127] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:03.272613 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1781b80-a678-4abd-a743-3d2eb357ca34 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:03.272808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1781b80-a678-4abd-a743-3d2eb357ca34 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:03.273430 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c417e0-7990-490a-b941-20ef021ec010 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:03.273610 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1781b80-a678-4abd-a743-3d2eb357ca34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:03.273889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1781b80-a678-4abd-a743-3d2eb357ca34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:03.275519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1781b80-a678-4abd-a743-3d2eb357ca34 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:03.284785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:03.286330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 64/128] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:03.289293 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1781b80-a678-4abd-a743-3d2eb357ca34 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:03.327910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:03.405917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c417e0-7990-490a-b941-20ef021ec010 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:03.421544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1781b80-a678-4abd-a743-3d2eb357ca34 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:03.450360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87135e0-6f4e-4ec1-a2b4-9a800336ff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:03.451158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87135e0-6f4e-4ec1-a2b4-9a800336ff39 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:03.451442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87135e0-6f4e-4ec1-a2b4-9a800336ff39 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:03.451715 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87135e0-6f4e-4ec1-a2b4-9a800336ff39 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:03.453497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87135e0-6f4e-4ec1-a2b4-9a800336ff39 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:03.455187 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87135e0-6f4e-4ec1-a2b4-9a800336ff39 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:03.457501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 65/129] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:03 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:03.458278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87135e0-6f4e-4ec1-a2b4-9a800336ff39 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:03.461651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcb418-4a89-497a-b204-37fd05cee213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:03.462567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcb418-4a89-497a-b204-37fd05cee213 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:03.463316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcb418-4a89-497a-b204-37fd05cee213 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:03.463742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcb418-4a89-497a-b204-37fd05cee213 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859/users/3300b0df5e5741c8b0d940d0202bb604/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:03.467840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dcb418-4a89-497a-b204-37fd05cee213 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61d2a2c3629481f80a0cb5f9c63f859, user_id=3300b0df5e5741c8b0d940d0202bb604, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:03.469560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dcb418-4a89-497a-b204-37fd05cee213 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:03.485369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 65/130] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:24:03 2022] PUT /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859/users/3300b0df5e5741c8b0d940d0202bb604/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:03.491839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcb418-4a89-497a-b204-37fd05cee213 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:03.550209 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:03.585166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87135e0-6f4e-4ec1-a2b4-9a800336ff39 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:03.612988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:03.621435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea236c8f-e01c-47c2-8b4e-869d903ecf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:03.622118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea236c8f-e01c-47c2-8b4e-869d903ecf4f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:03.622403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea236c8f-e01c-47c2-8b4e-869d903ecf4f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:03.622682 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea236c8f-e01c-47c2-8b4e-869d903ecf4f None admin] PATH_INFO: `/v3/users/0ff656a811aa4e698617e13e80ad9d73` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:03.623887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea236c8f-e01c-47c2-8b4e-869d903ecf4f None admin] RBAC: Authorizing `identity:get_user(user_id=0ff656a811aa4e698617e13e80ad9d73)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:03.625453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea236c8f-e01c-47c2-8b4e-869d903ecf4f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:03.628083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 66/131] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:03 2022] GET /identity/v3/users/0ff656a811aa4e698617e13e80ad9d73 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:03.628955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea236c8f-e01c-47c2-8b4e-869d903ecf4f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:03.629950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcb418-4a89-497a-b204-37fd05cee213 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:03.675438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc345c9-cb9f-45b6-94b4-62b3bb081fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:03.676108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc345c9-cb9f-45b6-94b4-62b3bb081fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:03.676399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc345c9-cb9f-45b6-94b4-62b3bb081fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:03.676680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc345c9-cb9f-45b6-94b4-62b3bb081fa8 None admin] PATH_INFO: `/v3/users/0ff656a811aa4e698617e13e80ad9d73` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:03.678472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc345c9-cb9f-45b6-94b4-62b3bb081fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=0ff656a811aa4e698617e13e80ad9d73)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:03.680713 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc345c9-cb9f-45b6-94b4-62b3bb081fa8 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:03.683294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 66/132] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:03 2022] GET /identity/v3/users/0ff656a811aa4e698617e13e80ad9d73 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:03.684830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba74e2-c500-4e17-8574-55e09645c4b8 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:03.685132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba74e2-c500-4e17-8574-55e09645c4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:03.685417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba74e2-c500-4e17-8574-55e09645c4b8 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:03.687358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 67/133] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:03.688072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba74e2-c500-4e17-8574-55e09645c4b8 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:03.761222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea236c8f-e01c-47c2-8b4e-869d903ecf4f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:03.808474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a7cb8-2c5e-4d89-a630-e980764b75b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:03.809175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a7cb8-2c5e-4d89-a630-e980764b75b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:03.809463 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a7cb8-2c5e-4d89-a630-e980764b75b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:03.809774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a7cb8-2c5e-4d89-a630-e980764b75b6 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859/users/0ff656a811aa4e698617e13e80ad9d73/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:03.812623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a7cb8-2c5e-4d89-a630-e980764b75b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61d2a2c3629481f80a0cb5f9c63f859, user_id=0ff656a811aa4e698617e13e80ad9d73, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:03.814837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a7cb8-2c5e-4d89-a630-e980764b75b6 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:03.820910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba74e2-c500-4e17-8574-55e09645c4b8 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:03.838341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 67/134] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:24:03 2022] PUT /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859/users/0ff656a811aa4e698617e13e80ad9d73/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:03.838942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a7cb8-2c5e-4d89-a630-e980764b75b6 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:03.848597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:03.875838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e20e8a-a790-40e5-b4ac-3d93379d7349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:03.877112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e20e8a-a790-40e5-b4ac-3d93379d7349 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:03.878693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e20e8a-a790-40e5-b4ac-3d93379d7349 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:03.879382 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e20e8a-a790-40e5-b4ac-3d93379d7349 None admin] PATH_INFO: `/v3/projects/bc3122d8dc20463bb81537175e4c5d28` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:03.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e20e8a-a790-40e5-b4ac-3d93379d7349 None admin] RBAC: Authorizing `identity:get_project(project_id=bc3122d8dc20463bb81537175e4c5d28)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:03.886735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e20e8a-a790-40e5-b4ac-3d93379d7349 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:03.890705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 68/135] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:03 2022] GET /identity/v3/projects/bc3122d8dc20463bb81537175e4c5d28 => generated 312 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:03.896089 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673b296-f790-4aaf-a1f7-cf4fbe74826f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:03.897199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673b296-f790-4aaf-a1f7-cf4fbe74826f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:03.898285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673b296-f790-4aaf-a1f7-cf4fbe74826f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:03.939645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b673b296-f790-4aaf-a1f7-cf4fbe74826f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:03.942015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b673b296-f790-4aaf-a1f7-cf4fbe74826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:03.989254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:04.050120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a7cb8-2c5e-4d89-a630-e980764b75b6 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:04.089563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 69/136] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:04.090310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673b296-f790-4aaf-a1f7-cf4fbe74826f None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:04.106863 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b26ae-10c1-4ac7-a438-d7b12210f5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:04.108200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b26ae-10c1-4ac7-a438-d7b12210f5ac None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:04.108790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b26ae-10c1-4ac7-a438-d7b12210f5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:04.109292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b26ae-10c1-4ac7-a438-d7b12210f5ac None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:04.110650 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b26ae-10c1-4ac7-a438-d7b12210f5ac None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:04.112873 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b26ae-10c1-4ac7-a438-d7b12210f5ac None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:04.115044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 68/137] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:03 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:04.126494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b26ae-10c1-4ac7-a438-d7b12210f5ac None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:04.209982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:04.243941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b26ae-10c1-4ac7-a438-d7b12210f5ac None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:04.259206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673b296-f790-4aaf-a1f7-cf4fbe74826f None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:04.279637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a99592-751d-4f4c-be47-1de6f6f98ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:04.280682 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a99592-751d-4f4c-be47-1de6f6f98ebf None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:04.281195 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a99592-751d-4f4c-be47-1de6f6f98ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:04.281672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a99592-751d-4f4c-be47-1de6f6f98ebf None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859/users/0ff656a811aa4e698617e13e80ad9d73/roles/89694ebde7824b8190bd5b15ece2b11e` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:04.284607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a99592-751d-4f4c-be47-1de6f6f98ebf None admin] RBAC: Authorizing `identity:create_grant(project_id=c61d2a2c3629481f80a0cb5f9c63f859, user_id=0ff656a811aa4e698617e13e80ad9d73, role_id=89694ebde7824b8190bd5b15ece2b11e)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:04.286296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a99592-751d-4f4c-be47-1de6f6f98ebf None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:04.302810 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 69/138] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:24:04 2022] PUT /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859/users/0ff656a811aa4e698617e13e80ad9d73/roles/89694ebde7824b8190bd5b15ece2b11e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:24:04.304783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e800396-4e8f-4e51-b341-9e8ab7890ff3 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:04.305296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e800396-4e8f-4e51-b341-9e8ab7890ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:04.305844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e800396-4e8f-4e51-b341-9e8ab7890ff3 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:04.307773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 70/139] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:04.319116 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7ef11-7039-4426-9437-b47629059aa6 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:04.319737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7ef11-7039-4426-9437-b47629059aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:04.320217 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7ef11-7039-4426-9437-b47629059aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:04.348552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-3fb7ef11-7039-4426-9437-b47629059aa6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:04.350250 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3fb7ef11-7039-4426-9437-b47629059aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:04.357347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2707f-aec5-4d47-897c-db5711251b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:04.357347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2707f-aec5-4d47-897c-db5711251b79 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:04.359512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2707f-aec5-4d47-897c-db5711251b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:04.359629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2707f-aec5-4d47-897c-db5711251b79 None admin] PATH_INFO: `/v3/projects/bc3122d8dc20463bb81537175e4c5d28/users/0ff656a811aa4e698617e13e80ad9d73/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:04.373719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2707f-aec5-4d47-897c-db5711251b79 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc3122d8dc20463bb81537175e4c5d28, user_id=0ff656a811aa4e698617e13e80ad9d73, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:04.373719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2707f-aec5-4d47-897c-db5711251b79 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:04.405729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 70/140] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:24:04 2022] PUT /identity/v3/projects/bc3122d8dc20463bb81537175e4c5d28/users/0ff656a811aa4e698617e13e80ad9d73/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:04.497722 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 71/141] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:04.500303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2707f-aec5-4d47-897c-db5711251b79 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:04.533961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:04.662265 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2707f-aec5-4d47-897c-db5711251b79 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:04.707857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993ae1f-f4d3-4293-b33c-544dc30330e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:04.708505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993ae1f-f4d3-4293-b33c-544dc30330e6 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:04.708783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993ae1f-f4d3-4293-b33c-544dc30330e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:04.709258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993ae1f-f4d3-4293-b33c-544dc30330e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:04.710955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993ae1f-f4d3-4293-b33c-544dc30330e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:04.713298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993ae1f-f4d3-4293-b33c-544dc30330e6 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:04.715890 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 71/142] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:04.723608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7ef11-7039-4426-9437-b47629059aa6 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:04.821281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:04.882921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7ef11-7039-4426-9437-b47629059aa6 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:04.904749 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993ae1f-f4d3-4293-b33c-544dc30330e6 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:04.931773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847778c-aa6d-46c7-8f40-d0e33aa1d41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:04.932328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847778c-aa6d-46c7-8f40-d0e33aa1d41c None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:04.932615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847778c-aa6d-46c7-8f40-d0e33aa1d41c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:04.933108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847778c-aa6d-46c7-8f40-d0e33aa1d41c None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:04.933777 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847778c-aa6d-46c7-8f40-d0e33aa1d41c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:04.935249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847778c-aa6d-46c7-8f40-d0e33aa1d41c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:04.974621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 72/143] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:24:04 2022] POST /identity/v3/users => generated 283 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:24:05.053618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:05.078124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993ae1f-f4d3-4293-b33c-544dc30330e6 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:05.130406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f297486-df78-4a18-b2b8-69bc37c8360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:05.131076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f297486-df78-4a18-b2b8-69bc37c8360d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:05.131357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f297486-df78-4a18-b2b8-69bc37c8360d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:05.131634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f297486-df78-4a18-b2b8-69bc37c8360d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:05.146732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f297486-df78-4a18-b2b8-69bc37c8360d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-9f297486-df78-4a18-b2b8-69bc37c8360d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:05.152514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:05.153519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:05.154294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 72/144] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:05.161992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847778c-aa6d-46c7-8f40-d0e33aa1d41c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:05.305016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847778c-aa6d-46c7-8f40-d0e33aa1d41c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:05.322856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:05.344938 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291523c-4c57-4bbe-9965-2ffa7ba4354d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:05.345700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291523c-4c57-4bbe-9965-2ffa7ba4354d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:05.345987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291523c-4c57-4bbe-9965-2ffa7ba4354d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:05.346264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291523c-4c57-4bbe-9965-2ffa7ba4354d None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:05.346958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291523c-4c57-4bbe-9965-2ffa7ba4354d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:05.347262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291523c-4c57-4bbe-9965-2ffa7ba4354d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:05.348671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0291523c-4c57-4bbe-9965-2ffa7ba4354d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:05.358562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 73/145] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:05.365331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f297486-df78-4a18-b2b8-69bc37c8360d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:05.480973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f297486-df78-4a18-b2b8-69bc37c8360d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:05.528043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0c2c6-b44d-41fb-829a-5ea27a3ece53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:05.528750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0c2c6-b44d-41fb-829a-5ea27a3ece53 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:05.528982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0c2c6-b44d-41fb-829a-5ea27a3ece53 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:05.529277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0c2c6-b44d-41fb-829a-5ea27a3ece53 None admin] PATH_INFO: `/v3/users/a06a548fe9a74214ba8da3f9ece2d6fb` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:05.537121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0c2c6-b44d-41fb-829a-5ea27a3ece53 None admin] RBAC: Authorizing `identity:get_user(user_id=a06a548fe9a74214ba8da3f9ece2d6fb)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:05.537121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0c2c6-b44d-41fb-829a-5ea27a3ece53 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:05.537121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 73/146] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:05 2022] GET /identity/v3/users/a06a548fe9a74214ba8da3f9ece2d6fb => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:05.543521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291523c-4c57-4bbe-9965-2ffa7ba4354d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:05.656896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291523c-4c57-4bbe-9965-2ffa7ba4354d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:05.696779 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8bbe7-4618-4b78-932e-975b873391a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:05.697534 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8bbe7-4618-4b78-932e-975b873391a8 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:05.697816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8bbe7-4618-4b78-932e-975b873391a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:05.698110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8bbe7-4618-4b78-932e-975b873391a8 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:05.699368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8bbe7-4618-4b78-932e-975b873391a8 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:05.700850 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8bbe7-4618-4b78-932e-975b873391a8 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:05.702837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 74/147] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:05 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:05.709066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0c2c6-b44d-41fb-829a-5ea27a3ece53 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:05.723401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:05.849730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0c2c6-b44d-41fb-829a-5ea27a3ece53 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:05.907736 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6c8b2-bb26-4b74-9341-9d2856689621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:05.908370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6c8b2-bb26-4b74-9341-9d2856689621 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:05.908638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6c8b2-bb26-4b74-9341-9d2856689621 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:05.908917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6c8b2-bb26-4b74-9341-9d2856689621 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859/users/a06a548fe9a74214ba8da3f9ece2d6fb/roles/718dd530c67f4a1891e292e729330f51` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:05.914494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6c8b2-bb26-4b74-9341-9d2856689621 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61d2a2c3629481f80a0cb5f9c63f859, user_id=a06a548fe9a74214ba8da3f9ece2d6fb, role_id=718dd530c67f4a1891e292e729330f51)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:05.915963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6c8b2-bb26-4b74-9341-9d2856689621 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:05.933440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 74/148] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:24:05 2022] PUT /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859/users/a06a548fe9a74214ba8da3f9ece2d6fb/roles/718dd530c67f4a1891e292e729330f51 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:07.118622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d70332-ff9a-4029-9d70-24bedf00f139 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:07.119131 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d70332-ff9a-4029-9d70-24bedf00f139 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:07.119233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d70332-ff9a-4029-9d70-24bedf00f139 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:07.120733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 75/149] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:07.131434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986653a-66e5-4d12-bbd5-b7c3a0e7c0c9 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:07.131951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986653a-66e5-4d12-bbd5-b7c3a0e7c0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:07.132042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986653a-66e5-4d12-bbd5-b7c3a0e7c0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:07.164975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-3986653a-66e5-4d12-bbd5-b7c3a0e7c0c9 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:07.166226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3986653a-66e5-4d12-bbd5-b7c3a0e7c0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:07.244530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:07.250437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 75/150] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:07.317066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29946dc3-acdf-44a2-8942-c88257813a98 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:07.317730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29946dc3-acdf-44a2-8942-c88257813a98 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:07.318215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29946dc3-acdf-44a2-8942-c88257813a98 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:07.320239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 76/151] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:07.327144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380bbcc-f8b9-40d8-b451-c35bbbddec62 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:07.327452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380bbcc-f8b9-40d8-b451-c35bbbddec62 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:07.327687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380bbcc-f8b9-40d8-b451-c35bbbddec62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:07.358287 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-6380bbcc-f8b9-40d8-b451-c35bbbddec62 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:07.359508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-6380bbcc-f8b9-40d8-b451-c35bbbddec62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:07.433869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 76/152] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:07.530783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fcf04-ce9d-494a-94e4-60f9d2f0d1e1 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:07.531381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fcf04-ce9d-494a-94e4-60f9d2f0d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:07.531855 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fcf04-ce9d-494a-94e4-60f9d2f0d1e1 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:07.533760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 77/153] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:07.538924 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6693fc3-b46c-45d4-b701-1febb394c5bf None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:07.539206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6693fc3-b46c-45d4-b701-1febb394c5bf None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:07.539478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6693fc3-b46c-45d4-b701-1febb394c5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:07.565781 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e6693fc3-b46c-45d4-b701-1febb394c5bf None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:07.567437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e6693fc3-b46c-45d4-b701-1febb394c5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:07.652816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 77/154] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:07.929733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fcf04-ce9d-494a-94e4-60f9d2f0d1e1 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:07.997175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:08.018186 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6693fc3-b46c-45d4-b701-1febb394c5bf None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:08.079392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fcf04-ce9d-494a-94e4-60f9d2f0d1e1 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:08.128370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:08.136553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb018-afe8-47b4-a21d-8606673e47d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:08.138212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb018-afe8-47b4-a21d-8606673e47d1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:08.138700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb018-afe8-47b4-a21d-8606673e47d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:08.139283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb018-afe8-47b4-a21d-8606673e47d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:08.158367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2cb018-afe8-47b4-a21d-8606673e47d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-1e2cb018-afe8-47b4-a21d-8606673e47d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:08.164455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:08.165327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:08.166078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:08.166078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:08.166078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:08.166078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:08.166078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:08.166078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:08.166078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:08.166078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:08.166078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:08.166078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:08.166078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:08.166078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:08.168397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 78/155] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:24:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:08.174329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb018-afe8-47b4-a21d-8606673e47d1 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:08.227322 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6693fc3-b46c-45d4-b701-1febb394c5bf None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:08.315735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb018-afe8-47b4-a21d-8606673e47d1 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:08.337123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:08.342009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f85cb-c1b5-4ced-ba47-7f18e27cacd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:08.342714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f85cb-c1b5-4ced-ba47-7f18e27cacd8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:08.343284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f85cb-c1b5-4ced-ba47-7f18e27cacd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:08.343284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f85cb-c1b5-4ced-ba47-7f18e27cacd8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:08.372694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f85cb-c1b5-4ced-ba47-7f18e27cacd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-6d9f85cb-c1b5-4ced-ba47-7f18e27cacd8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:08.378964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:08.379845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:08.380853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 78/156] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:08.381541 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f85cb-c1b5-4ced-ba47-7f18e27cacd8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:08.386559 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac59053-3e9d-43c0-b8a3-91c865567352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:08.387522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac59053-3e9d-43c0-b8a3-91c865567352 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:08.388020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac59053-3e9d-43c0-b8a3-91c865567352 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:08.388550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac59053-3e9d-43c0-b8a3-91c865567352 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:08.389493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac59053-3e9d-43c0-b8a3-91c865567352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:08.390011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac59053-3e9d-43c0-b8a3-91c865567352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:08.391742 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac59053-3e9d-43c0-b8a3-91c865567352 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:08.407222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 79/157] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:08.409000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac59053-3e9d-43c0-b8a3-91c865567352 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:08.488220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f85cb-c1b5-4ced-ba47-7f18e27cacd8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:08.515781 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:08.539407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac69b77-94a1-4fbb-8e36-9eecb03a03af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:08.540124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac69b77-94a1-4fbb-8e36-9eecb03a03af None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:08.540397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac69b77-94a1-4fbb-8e36-9eecb03a03af None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:08.540748 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac69b77-94a1-4fbb-8e36-9eecb03a03af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:08.559142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac69b77-94a1-4fbb-8e36-9eecb03a03af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-cac69b77-94a1-4fbb-8e36-9eecb03a03af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:08.564113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:08.565756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:08.566766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:08.566766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:08.566766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:08.566766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:08.566766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:08.566766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:08.566766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:08.566766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:08.566766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:08.566766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:08.566766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:08.566766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:08.574258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 79/158] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:24:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:08.575232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac69b77-94a1-4fbb-8e36-9eecb03a03af None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:08.662062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac59053-3e9d-43c0-b8a3-91c865567352 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:08.774488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e1bb4-9bbd-45eb-8c49-f0526b43f98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:08.775044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e1bb4-9bbd-45eb-8c49-f0526b43f98d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:08.775583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e1bb4-9bbd-45eb-8c49-f0526b43f98d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:08.775861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e1bb4-9bbd-45eb-8c49-f0526b43f98d None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:08.776544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e1bb4-9bbd-45eb-8c49-f0526b43f98d None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:08.776851 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e1bb4-9bbd-45eb-8c49-f0526b43f98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:08.778301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e1bb4-9bbd-45eb-8c49-f0526b43f98d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:08.799895 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 80/159] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:08.800796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e1bb4-9bbd-45eb-8c49-f0526b43f98d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:08.843089 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac69b77-94a1-4fbb-8e36-9eecb03a03af None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:08.876530 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:08.923770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca0ef9-d650-4498-a92a-9a08429f6f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:08.924429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca0ef9-d650-4498-a92a-9a08429f6f09 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:08.924712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca0ef9-d650-4498-a92a-9a08429f6f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:08.924999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca0ef9-d650-4498-a92a-9a08429f6f09 None admin] PATH_INFO: `/v3/users/3300b0df5e5741c8b0d940d0202bb604` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:08.931226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca0ef9-d650-4498-a92a-9a08429f6f09 None admin] RBAC: Authorizing `identity:get_user(user_id=3300b0df5e5741c8b0d940d0202bb604)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:08.931226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca0ef9-d650-4498-a92a-9a08429f6f09 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:08.931597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 80/160] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:08 2022] GET /identity/v3/users/3300b0df5e5741c8b0d940d0202bb604 => generated 247 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:08.932644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daad5a5-0c07-4f8d-8af1-4719a6ddce6f None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:08.932918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daad5a5-0c07-4f8d-8af1-4719a6ddce6f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:08.933214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daad5a5-0c07-4f8d-8af1-4719a6ddce6f None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:08.935316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 81/161] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:08.936109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daad5a5-0c07-4f8d-8af1-4719a6ddce6f None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:09.055372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e1bb4-9bbd-45eb-8c49-f0526b43f98d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:09.071599 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:09.125975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77442fc8-4f07-48c4-88bf-47c95ea616a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:09.126591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77442fc8-4f07-48c4-88bf-47c95ea616a7 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:09.126914 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77442fc8-4f07-48c4-88bf-47c95ea616a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:09.127194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77442fc8-4f07-48c4-88bf-47c95ea616a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:09.127975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77442fc8-4f07-48c4-88bf-47c95ea616a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:09.128295 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77442fc8-4f07-48c4-88bf-47c95ea616a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:09.129722 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77442fc8-4f07-48c4-88bf-47c95ea616a7 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:09.143415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daad5a5-0c07-4f8d-8af1-4719a6ddce6f None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:09.151585 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 81/162] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:09.152717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77442fc8-4f07-48c4-88bf-47c95ea616a7 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:09.201328 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbd1c8-1656-473d-800a-1aab2e549302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:09.202329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbd1c8-1656-473d-800a-1aab2e549302 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:09.202946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbd1c8-1656-473d-800a-1aab2e549302 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:09.203426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbd1c8-1656-473d-800a-1aab2e549302 None admin] PATH_INFO: `/v3/users/0ff656a811aa4e698617e13e80ad9d73` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:09.205400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbd1c8-1656-473d-800a-1aab2e549302 None admin] RBAC: Authorizing `identity:get_user(user_id=0ff656a811aa4e698617e13e80ad9d73)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:09.207236 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbd1c8-1656-473d-800a-1aab2e549302 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:09.217646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 82/163] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:08 2022] GET /identity/v3/users/0ff656a811aa4e698617e13e80ad9d73 => generated 275 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:09.219588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c9ffc-3006-4fa1-ad5a-a788b2609f2f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:09.220098 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c9ffc-3006-4fa1-ad5a-a788b2609f2f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:09.221180 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c9ffc-3006-4fa1-ad5a-a788b2609f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:09.246191 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-fb7c9ffc-3006-4fa1-ad5a-a788b2609f2f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:09.248313 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-fb7c9ffc-3006-4fa1-ad5a-a788b2609f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:09.379007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 83/164] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:09.379925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c9ffc-3006-4fa1-ad5a-a788b2609f2f None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:09.409484 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:09.440297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77442fc8-4f07-48c4-88bf-47c95ea616a7 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:09.480099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:09.562529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cab992-7c6a-47a5-8452-15c36349c424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:09.563233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cab992-7c6a-47a5-8452-15c36349c424 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:09.563545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cab992-7c6a-47a5-8452-15c36349c424 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:09.563819 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cab992-7c6a-47a5-8452-15c36349c424 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:09.628016 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c9ffc-3006-4fa1-ad5a-a788b2609f2f None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:09.709841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb77a7-71e5-4208-b169-5ca6a268fcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:09.710550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb77a7-71e5-4208-b169-5ca6a268fcf2 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:09.710834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb77a7-71e5-4208-b169-5ca6a268fcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:09.711107 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb77a7-71e5-4208-b169-5ca6a268fcf2 None admin] PATH_INFO: `/v3/users/0ff656a811aa4e698617e13e80ad9d73` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:09.713611 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb77a7-71e5-4208-b169-5ca6a268fcf2 None admin] RBAC: Authorizing `identity:get_user(user_id=0ff656a811aa4e698617e13e80ad9d73)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:09.715228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb77a7-71e5-4208-b169-5ca6a268fcf2 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:09.717172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 84/165] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:09 2022] GET /identity/v3/users/0ff656a811aa4e698617e13e80ad9d73 => generated 275 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:09.718288 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb77a7-71e5-4208-b169-5ca6a268fcf2 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:09.756404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cab992-7c6a-47a5-8452-15c36349c424 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:09.758446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cab992-7c6a-47a5-8452-15c36349c424 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:09.922341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb77a7-71e5-4208-b169-5ca6a268fcf2 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:09.953617 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:09.973545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:09.993630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 82/166] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 841 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:09.993968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21daaec-32d1-4ed6-8fa8-938ab8675436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:09.994611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cab992-7c6a-47a5-8452-15c36349c424 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:09.994722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21daaec-32d1-4ed6-8fa8-938ab8675436 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:09.994946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21daaec-32d1-4ed6-8fa8-938ab8675436 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:09.995300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21daaec-32d1-4ed6-8fa8-938ab8675436 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:10.079571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21daaec-32d1-4ed6-8fa8-938ab8675436 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:10.081196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21daaec-32d1-4ed6-8fa8-938ab8675436 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:10.186103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cab992-7c6a-47a5-8452-15c36349c424 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:10.208209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 85/167] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:10.210533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6664ecd-f8c4-407e-af17-19a4bb27d416 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:10.211038 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6664ecd-f8c4-407e-af17-19a4bb27d416 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:10.211546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6664ecd-f8c4-407e-af17-19a4bb27d416 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:10.218822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 86/168] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:10.220318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6664ecd-f8c4-407e-af17-19a4bb27d416 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:10.254161 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:10.271911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058ca21-9f27-44c3-9646-eeee426f7db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:10.272548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058ca21-9f27-44c3-9646-eeee426f7db3 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:10.272829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058ca21-9f27-44c3-9646-eeee426f7db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:10.273121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058ca21-9f27-44c3-9646-eeee426f7db3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:10.348679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:10.363455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058ca21-9f27-44c3-9646-eeee426f7db3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:10.365724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058ca21-9f27-44c3-9646-eeee426f7db3 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:10.386780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6664ecd-f8c4-407e-af17-19a4bb27d416 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:10.435220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86333aca-3b23-4c83-93a3-609031b17b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:10.435864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86333aca-3b23-4c83-93a3-609031b17b04 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:10.436196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86333aca-3b23-4c83-93a3-609031b17b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:10.436478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86333aca-3b23-4c83-93a3-609031b17b04 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:10.437923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86333aca-3b23-4c83-93a3-609031b17b04 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:10.439421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86333aca-3b23-4c83-93a3-609031b17b04 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:10.445212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 87/169] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:10 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:10.445212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86333aca-3b23-4c83-93a3-609031b17b04 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:10.489706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 83/170] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:10.494668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed708cac-0ed5-43cb-a86d-27585c6cddfa None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:10.494758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed708cac-0ed5-43cb-a86d-27585c6cddfa None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:10.495012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed708cac-0ed5-43cb-a86d-27585c6cddfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:10.513300 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ed708cac-0ed5-43cb-a86d-27585c6cddfa None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:10.514783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ed708cac-0ed5-43cb-a86d-27585c6cddfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:10.550863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:10.572369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86333aca-3b23-4c83-93a3-609031b17b04 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:10.587215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 84/171] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:10.588100 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed708cac-0ed5-43cb-a86d-27585c6cddfa None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:10.625835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9505139-95aa-4a43-ac08-0095c3bd1d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:10.626489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9505139-95aa-4a43-ac08-0095c3bd1d80 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:10.626840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9505139-95aa-4a43-ac08-0095c3bd1d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:10.627625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9505139-95aa-4a43-ac08-0095c3bd1d80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:10.628861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:10.639771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9505139-95aa-4a43-ac08-0095c3bd1d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-a9505139-95aa-4a43-ac08-0095c3bd1d80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:10.645212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:10.646146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:10.646910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:10.646910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:10.646910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:10.646910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:10.646910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:10.646910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:10.646910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:10.646910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:10.646910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:10.646910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:10.646910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:10.646910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:10.648970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 88/172] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:10.650590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea53ad0-6ffa-4d1c-99c2-2fd9f4440e50 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:10.650879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea53ad0-6ffa-4d1c-99c2-2fd9f4440e50 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:10.651155 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea53ad0-6ffa-4d1c-99c2-2fd9f4440e50 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:10.652528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 89/173] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:10.653643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443d9f0-50dc-4441-ad5f-52f371f62fcf None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:10.653924 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443d9f0-50dc-4441-ad5f-52f371f62fcf None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:10.654199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443d9f0-50dc-4441-ad5f-52f371f62fcf None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:10.655794 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 90/174] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:10.656612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443d9f0-50dc-4441-ad5f-52f371f62fcf None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:10.742129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed708cac-0ed5-43cb-a86d-27585c6cddfa None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:10.778618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443d9f0-50dc-4441-ad5f-52f371f62fcf None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:10.789036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb20c2-9c03-4798-b846-447805b69d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:10.790042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb20c2-9c03-4798-b846-447805b69d37 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:10.790585 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb20c2-9c03-4798-b846-447805b69d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:10.791080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb20c2-9c03-4798-b846-447805b69d37 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:10.792440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb20c2-9c03-4798-b846-447805b69d37 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:10.794360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb20c2-9c03-4798-b846-447805b69d37 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:10.796299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 85/175] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:10 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:10.797080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb20c2-9c03-4798-b846-447805b69d37 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:10.822648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b1184-1ef5-4ebe-bf86-21eaf85c7686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:10.823254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b1184-1ef5-4ebe-bf86-21eaf85c7686 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:10.823527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b1184-1ef5-4ebe-bf86-21eaf85c7686 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:10.823794 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b1184-1ef5-4ebe-bf86-21eaf85c7686 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:10.824522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b1184-1ef5-4ebe-bf86-21eaf85c7686 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859, user.id=3300b0df5e5741c8b0d940d0202bb604) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:10.824825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b1184-1ef5-4ebe-bf86-21eaf85c7686 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:10.826319 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b1184-1ef5-4ebe-bf86-21eaf85c7686 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:10.833341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 91/176] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:10 2022] GET /identity/v3/role_assignments?user.id=3300b0df5e5741c8b0d940d0202bb604&scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:10.834532 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2582da9-6de9-4f7a-87e7-c4e97604b51b None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:10.834815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2582da9-6de9-4f7a-87e7-c4e97604b51b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:10.835084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2582da9-6de9-4f7a-87e7-c4e97604b51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:10.857490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:10.861580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f2582da9-6de9-4f7a-87e7-c4e97604b51b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:10.863218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f2582da9-6de9-4f7a-87e7-c4e97604b51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:10.928507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:10.942750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb20c2-9c03-4798-b846-447805b69d37 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:10.958671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 92/177] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:10.959380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2582da9-6de9-4f7a-87e7-c4e97604b51b None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:11.000732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c3348-a8fe-4ca3-a0bc-115af8dfb419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:11.001472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c3348-a8fe-4ca3-a0bc-115af8dfb419 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:11.002290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c3348-a8fe-4ca3-a0bc-115af8dfb419 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:11.002560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c3348-a8fe-4ca3-a0bc-115af8dfb419 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:11.004214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c3348-a8fe-4ca3-a0bc-115af8dfb419 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:11.005459 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c3348-a8fe-4ca3-a0bc-115af8dfb419 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:11.007222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 86/178] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:10 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:11.013658 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b92d25-8fee-4eff-b148-901d7277f1d0 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:11.013984 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b92d25-8fee-4eff-b148-901d7277f1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:11.014271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b92d25-8fee-4eff-b148-901d7277f1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:11.037948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-b3b92d25-8fee-4eff-b148-901d7277f1d0 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:11.039683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b3b92d25-8fee-4eff-b148-901d7277f1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:11.110654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2582da9-6de9-4f7a-87e7-c4e97604b51b None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:11.150863 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 87/179] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:11.154020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600596d5-9edd-476b-aae0-b56dda80b092 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:11.154512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600596d5-9edd-476b-aae0-b56dda80b092 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:11.155205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600596d5-9edd-476b-aae0-b56dda80b092 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:11.158306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 88/180] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:11.159307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600596d5-9edd-476b-aae0-b56dda80b092 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:11.162561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd747c-f22d-4b0d-82a6-e46deed8d507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:11.163160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd747c-f22d-4b0d-82a6-e46deed8d507 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:11.163237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd747c-f22d-4b0d-82a6-e46deed8d507 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:11.163578 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd747c-f22d-4b0d-82a6-e46deed8d507 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:11.164598 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd747c-f22d-4b0d-82a6-e46deed8d507 None admin] RBAC: Adding query filter params (name=member) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:11.164969 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd747c-f22d-4b0d-82a6-e46deed8d507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:11.167464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd747c-f22d-4b0d-82a6-e46deed8d507 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:11.170504 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:11.180463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 93/181] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:11.185139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd747c-f22d-4b0d-82a6-e46deed8d507 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:11.276517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:11.292591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600596d5-9edd-476b-aae0-b56dda80b092 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:11.314625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd747c-f22d-4b0d-82a6-e46deed8d507 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:11.329395 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69280d-8d5e-4dd5-aa92-b061d069839f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:11.330853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69280d-8d5e-4dd5-aa92-b061d069839f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:11.331124 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69280d-8d5e-4dd5-aa92-b061d069839f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:11.331393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69280d-8d5e-4dd5-aa92-b061d069839f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:11.332166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69280d-8d5e-4dd5-aa92-b061d069839f None admin] RBAC: Adding query filter params (role.id=4c80373ee4c34171a7f29e5ab6c09ce0, scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859, user.id=0ff656a811aa4e698617e13e80ad9d73) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:11.332557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69280d-8d5e-4dd5-aa92-b061d069839f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:11.334250 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69280d-8d5e-4dd5-aa92-b061d069839f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:11.349125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 89/182] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:11 2022] GET /identity/v3/role_assignments?user.id=0ff656a811aa4e698617e13e80ad9d73&scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859&role.id=4c80373ee4c34171a7f29e5ab6c09ce0 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:11.349125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69280d-8d5e-4dd5-aa92-b061d069839f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:11.368335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca5d54-4261-4ade-80bc-9175c91dee75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:11.369792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca5d54-4261-4ade-80bc-9175c91dee75 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:11.370605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca5d54-4261-4ade-80bc-9175c91dee75 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:11.371043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca5d54-4261-4ade-80bc-9175c91dee75 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:11.385909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca5d54-4261-4ade-80bc-9175c91dee75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-deca5d54-4261-4ade-80bc-9175c91dee75 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:11.392326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:11.393421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:11.394256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:11.394256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:11.394256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:11.394256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:11.394256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:11.394256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:11.394256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:11.394256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:11.394256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:11.394256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:11.394256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:11.394256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:11.396865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 94/183] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:11.399237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45990b17-0174-46da-95b8-3c5451e28401 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:11.399810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45990b17-0174-46da-95b8-3c5451e28401 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:11.400481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45990b17-0174-46da-95b8-3c5451e28401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:11.426833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-45990b17-0174-46da-95b8-3c5451e28401 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:11.428332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-45990b17-0174-46da-95b8-3c5451e28401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:11.503718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69280d-8d5e-4dd5-aa92-b061d069839f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:11.506014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:11.529419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 95/184] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:11.530325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45990b17-0174-46da-95b8-3c5451e28401 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:11.560897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d7a26-c5de-4d5b-8f3a-e200f43e9777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:11.560897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d7a26-c5de-4d5b-8f3a-e200f43e9777 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:11.560897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d7a26-c5de-4d5b-8f3a-e200f43e9777 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:11.560897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d7a26-c5de-4d5b-8f3a-e200f43e9777 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:11.561474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d7a26-c5de-4d5b-8f3a-e200f43e9777 None admin] RBAC: Adding query filter params (role.id=89694ebde7824b8190bd5b15ece2b11e, scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859, user.id=0ff656a811aa4e698617e13e80ad9d73) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:11.562357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d7a26-c5de-4d5b-8f3a-e200f43e9777 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:11.563329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d7a26-c5de-4d5b-8f3a-e200f43e9777 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:11.570157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 90/185] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:11 2022] GET /identity/v3/role_assignments?user.id=0ff656a811aa4e698617e13e80ad9d73&scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859&role.id=89694ebde7824b8190bd5b15ece2b11e => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:11.570858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d7a26-c5de-4d5b-8f3a-e200f43e9777 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:11.602846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:11.720828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45990b17-0174-46da-95b8-3c5451e28401 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:11.724369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d7a26-c5de-4d5b-8f3a-e200f43e9777 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:11.773008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff5c25-38ce-4adc-a53c-7838fdfac61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:11.774091 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff5c25-38ce-4adc-a53c-7838fdfac61f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:11.774576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff5c25-38ce-4adc-a53c-7838fdfac61f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:11.775062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff5c25-38ce-4adc-a53c-7838fdfac61f None admin] PATH_INFO: `/v3/users/0ff656a811aa4e698617e13e80ad9d73` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:11.776741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff5c25-38ce-4adc-a53c-7838fdfac61f None admin] RBAC: Authorizing `identity:get_user(user_id=0ff656a811aa4e698617e13e80ad9d73)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:11.777676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0d8fd-1bb6-4460-a829-9c6d6dac11ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:11.778330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0d8fd-1bb6-4460-a829-9c6d6dac11ff None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:11.778613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0d8fd-1bb6-4460-a829-9c6d6dac11ff None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:11.778888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0d8fd-1bb6-4460-a829-9c6d6dac11ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:11.779379 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff5c25-38ce-4adc-a53c-7838fdfac61f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:11.781634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 96/186] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:11 2022] GET /identity/v3/users/0ff656a811aa4e698617e13e80ad9d73 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:11.783123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff5c25-38ce-4adc-a53c-7838fdfac61f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:11.792550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0d8fd-1bb6-4460-a829-9c6d6dac11ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-3ec0d8fd-1bb6-4460-a829-9c6d6dac11ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:11.799381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:11.800375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 91/187] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:24:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:11.805253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0d8fd-1bb6-4460-a829-9c6d6dac11ff None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:11.860346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:11.890285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:11.912291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff5c25-38ce-4adc-a53c-7838fdfac61f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:11.915041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0d8fd-1bb6-4460-a829-9c6d6dac11ff None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:11.946500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277416c-bc45-4346-811d-85c7e35c25b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:11.947100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277416c-bc45-4346-811d-85c7e35c25b1 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:11.947379 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277416c-bc45-4346-811d-85c7e35c25b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:11.947641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277416c-bc45-4346-811d-85c7e35c25b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:11.948305 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277416c-bc45-4346-811d-85c7e35c25b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:11.948603 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277416c-bc45-4346-811d-85c7e35c25b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:11.948694 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffc8ab-d63b-498b-a433-55ddf786e02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:11.949356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffc8ab-d63b-498b-a433-55ddf786e02c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:11.949637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffc8ab-d63b-498b-a433-55ddf786e02c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:11.949910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ffc8ab-d63b-498b-a433-55ddf786e02c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:11.949996 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277416c-bc45-4346-811d-85c7e35c25b1 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:11.951046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ffc8ab-d63b-498b-a433-55ddf786e02c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:11.952429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ffc8ab-d63b-498b-a433-55ddf786e02c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:11.954014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 92/188] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:11.957969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffc8ab-d63b-498b-a433-55ddf786e02c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:11.958934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 97/189] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:11.959620 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277416c-bc45-4346-811d-85c7e35c25b1 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:12.060934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277416c-bc45-4346-811d-85c7e35c25b1 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:12.062848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ffc8ab-d63b-498b-a433-55ddf786e02c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:12.078224 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:12.095876 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebda26b-e9cd-45cf-9d16-d54b3ec49ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:12.096486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebda26b-e9cd-45cf-9d16-d54b3ec49ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:12.096768 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebda26b-e9cd-45cf-9d16-d54b3ec49ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:12.097045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebda26b-e9cd-45cf-9d16-d54b3ec49ca0 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:12.097756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebda26b-e9cd-45cf-9d16-d54b3ec49ca0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:12.097984 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b19e7-8751-4c8d-a690-f63c712c7f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:12.098089 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebda26b-e9cd-45cf-9d16-d54b3ec49ca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:12.098639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b19e7-8751-4c8d-a690-f63c712c7f34 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:12.098919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b19e7-8751-4c8d-a690-f63c712c7f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:12.099187 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b19e7-8751-4c8d-a690-f63c712c7f34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:12.099513 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebda26b-e9cd-45cf-9d16-d54b3ec49ca0 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:12.113234 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 98/190] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:12.113234 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebda26b-e9cd-45cf-9d16-d54b3ec49ca0 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:12.115072 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:12.178501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b19e7-8751-4c8d-a690-f63c712c7f34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:12.180277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b19e7-8751-4c8d-a690-f63c712c7f34 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:12.229751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebda26b-e9cd-45cf-9d16-d54b3ec49ca0 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:12.269856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ae084-c01f-4a2d-81c2-65d2902e6547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:12.270687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ae084-c01f-4a2d-81c2-65d2902e6547 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:12.271043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ae084-c01f-4a2d-81c2-65d2902e6547 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:12.271837 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ae084-c01f-4a2d-81c2-65d2902e6547 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:12.284644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ae084-c01f-4a2d-81c2-65d2902e6547 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-7d0ae084-c01f-4a2d-81c2-65d2902e6547 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:12.290184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:12.291069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:12.291859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:12.291859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:12.291859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:12.291859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:12.291859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:12.291859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:12.291859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:12.291859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:12.291859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:12.291859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:12.291859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:12.291859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:12.296509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 99/191] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:24:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:12.297416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ae084-c01f-4a2d-81c2-65d2902e6547 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:12.305439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 93/192] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:12.306831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b19e7-8751-4c8d-a690-f63c712c7f34 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:12.344627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:12.364801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:12.423149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b19e7-8751-4c8d-a690-f63c712c7f34 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:12.425579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ae084-c01f-4a2d-81c2-65d2902e6547 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:12.462612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cef289-f581-47f2-aff1-341b2a45ee25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:12.463374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cef289-f581-47f2-aff1-341b2a45ee25 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:12.463718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cef289-f581-47f2-aff1-341b2a45ee25 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:12.464054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cef289-f581-47f2-aff1-341b2a45ee25 None admin] PATH_INFO: `/v3/users/a06a548fe9a74214ba8da3f9ece2d6fb` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:12.464649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc5b35-c27d-4637-bab2-4f746dce55f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:12.465278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cef289-f581-47f2-aff1-341b2a45ee25 None admin] RBAC: Authorizing `identity:get_user(user_id=a06a548fe9a74214ba8da3f9ece2d6fb)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:12.465401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc5b35-c27d-4637-bab2-4f746dce55f2 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:12.465552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc5b35-c27d-4637-bab2-4f746dce55f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:12.465824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc5b35-c27d-4637-bab2-4f746dce55f2 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:12.466467 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc5b35-c27d-4637-bab2-4f746dce55f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:12.466723 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cef289-f581-47f2-aff1-341b2a45ee25 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:12.467870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc5b35-c27d-4637-bab2-4f746dce55f2 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:12.468491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 94/193] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:12 2022] GET /identity/v3/users/a06a548fe9a74214ba8da3f9ece2d6fb => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:12.469039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cef289-f581-47f2-aff1-341b2a45ee25 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:12.491767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 100/194] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:24:12 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:24:12.493074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc5b35-c27d-4637-bab2-4f746dce55f2 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:12.600626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cef289-f581-47f2-aff1-341b2a45ee25 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:12.617563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:12.624573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:12.633296 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc5b35-c27d-4637-bab2-4f746dce55f2 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:12.647262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66fee2-2f2f-4ad1-89fa-adfb19a5167b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:12.647947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66fee2-2f2f-4ad1-89fa-adfb19a5167b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:12.648200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66fee2-2f2f-4ad1-89fa-adfb19a5167b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:12.648472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66fee2-2f2f-4ad1-89fa-adfb19a5167b None admin] PATH_INFO: `/v3/users/3c7acd70496347beb8e901f1fbb8d4d0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:12.665611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66fee2-2f2f-4ad1-89fa-adfb19a5167b None admin] RBAC: Authorizing `identity:get_user(user_id=3c7acd70496347beb8e901f1fbb8d4d0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:12.666472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66fee2-2f2f-4ad1-89fa-adfb19a5167b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:12.668504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 95/195] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:12 2022] GET /identity/v3/users/3c7acd70496347beb8e901f1fbb8d4d0 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:12.674287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66fee2-2f2f-4ad1-89fa-adfb19a5167b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:12.683949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567fd69-1d0c-4d94-b606-849b76497579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:12.684568 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567fd69-1d0c-4d94-b606-849b76497579 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:12.684847 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567fd69-1d0c-4d94-b606-849b76497579 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:12.685759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567fd69-1d0c-4d94-b606-849b76497579 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:12.685866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567fd69-1d0c-4d94-b606-849b76497579 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:12.686138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567fd69-1d0c-4d94-b606-849b76497579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:12.687548 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567fd69-1d0c-4d94-b606-849b76497579 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:12.699107 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 101/196] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:12.699784 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567fd69-1d0c-4d94-b606-849b76497579 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:12.821003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66fee2-2f2f-4ad1-89fa-adfb19a5167b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:12.833920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567fd69-1d0c-4d94-b606-849b76497579 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:12.868031 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ce441-8d63-4e06-b19a-8f192bbbf831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:12.868904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ce441-8d63-4e06-b19a-8f192bbbf831 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:12.869620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ce441-8d63-4e06-b19a-8f192bbbf831 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:12.870248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ce441-8d63-4e06-b19a-8f192bbbf831 None admin] PATH_INFO: `/v3/projects/bc3122d8dc20463bb81537175e4c5d28` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:12.871306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ce441-8d63-4e06-b19a-8f192bbbf831 None admin] RBAC: Authorizing `identity:get_project(project_id=bc3122d8dc20463bb81537175e4c5d28)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:12.872770 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ce441-8d63-4e06-b19a-8f192bbbf831 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:12.874647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8fa83-341e-4625-8d72-73e9bc6a64cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:12.874790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 96/197] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:12 2022] GET /identity/v3/projects/bc3122d8dc20463bb81537175e4c5d28 => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:12.875156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8fa83-341e-4625-8d72-73e9bc6a64cb None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:12.875258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ce441-8d63-4e06-b19a-8f192bbbf831 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:12.875436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8fa83-341e-4625-8d72-73e9bc6a64cb None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:12.875705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8fa83-341e-4625-8d72-73e9bc6a64cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:12.912128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:12.922284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:12.956986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8fa83-341e-4625-8d72-73e9bc6a64cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:12.958502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8fa83-341e-4625-8d72-73e9bc6a64cb None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:13.004862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ce441-8d63-4e06-b19a-8f192bbbf831 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:13.054409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6fd68-133a-421d-9d9c-bcacf24ccaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:13.055428 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6fd68-133a-421d-9d9c-bcacf24ccaf6 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:13.055937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6fd68-133a-421d-9d9c-bcacf24ccaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:13.056412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6fd68-133a-421d-9d9c-bcacf24ccaf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:13.076479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 102/198] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:13.077734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8fa83-341e-4625-8d72-73e9bc6a64cb None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:13.130240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6fd68-133a-421d-9d9c-bcacf24ccaf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:13.132043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6fd68-133a-421d-9d9c-bcacf24ccaf6 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:13.154105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:13.181436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:13.216061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8fa83-341e-4625-8d72-73e9bc6a64cb None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:13.241369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 97/199] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:13.242597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6fd68-133a-421d-9d9c-bcacf24ccaf6 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:13.262838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b74e7-daf0-4bb4-8cda-cccf7cb6a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:13.263473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b74e7-daf0-4bb4-8cda-cccf7cb6a62d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:13.263749 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b74e7-daf0-4bb4-8cda-cccf7cb6a62d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:13.264020 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b74e7-daf0-4bb4-8cda-cccf7cb6a62d None admin] PATH_INFO: `/v3/users/3c7acd70496347beb8e901f1fbb8d4d0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:13.265187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b74e7-daf0-4bb4-8cda-cccf7cb6a62d None admin] RBAC: Authorizing `identity:get_user(user_id=3c7acd70496347beb8e901f1fbb8d4d0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:13.267479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b74e7-daf0-4bb4-8cda-cccf7cb6a62d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:13.269419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 103/200] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:13 2022] GET /identity/v3/users/3c7acd70496347beb8e901f1fbb8d4d0 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:13.270941 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b74e7-daf0-4bb4-8cda-cccf7cb6a62d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:13.351366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6fd68-133a-421d-9d9c-bcacf24ccaf6 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:13.388268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257cb52-6087-4f4b-a573-34f6f58b8f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:13.389274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257cb52-6087-4f4b-a573-34f6f58b8f9b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:13.389731 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257cb52-6087-4f4b-a573-34f6f58b8f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:13.390179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257cb52-6087-4f4b-a573-34f6f58b8f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:13.391078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e257cb52-6087-4f4b-a573-34f6f58b8f9b None admin] RBAC: Adding query filter params (role.id=4c80373ee4c34171a7f29e5ab6c09ce0, scope.project.id=bc3122d8dc20463bb81537175e4c5d28, user.id=0ff656a811aa4e698617e13e80ad9d73) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:13.391771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e257cb52-6087-4f4b-a573-34f6f58b8f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:13.394075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e257cb52-6087-4f4b-a573-34f6f58b8f9b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:13.400826 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 98/201] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:13 2022] GET /identity/v3/role_assignments?user.id=0ff656a811aa4e698617e13e80ad9d73&scope.project.id=bc3122d8dc20463bb81537175e4c5d28&role.id=4c80373ee4c34171a7f29e5ab6c09ce0 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:13.402118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257cb52-6087-4f4b-a573-34f6f58b8f9b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:13.405265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:13.406490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b74e7-daf0-4bb4-8cda-cccf7cb6a62d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:13.447231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:13.461779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d889aeb-78b3-4308-a94a-62a62f1becfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:13.463429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d889aeb-78b3-4308-a94a-62a62f1becfa None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:13.463909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d889aeb-78b3-4308-a94a-62a62f1becfa None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:13.464480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d889aeb-78b3-4308-a94a-62a62f1becfa None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:13.465889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d889aeb-78b3-4308-a94a-62a62f1becfa None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:13.468477 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d889aeb-78b3-4308-a94a-62a62f1becfa None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:13.472220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 104/202] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:13 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:13.473776 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d889aeb-78b3-4308-a94a-62a62f1becfa None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:13.585917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257cb52-6087-4f4b-a573-34f6f58b8f9b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:13.615923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d889aeb-78b3-4308-a94a-62a62f1becfa None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:13.659074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a5583-4dfc-4d02-bc89-89abf43a1376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:13.659622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a5583-4dfc-4d02-bc89-89abf43a1376 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:13.659900 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a5583-4dfc-4d02-bc89-89abf43a1376 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:13.660170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a5583-4dfc-4d02-bc89-89abf43a1376 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:13.667108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9befdb-ff1d-4eaf-a62c-1d24a988572e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:13.667768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9befdb-ff1d-4eaf-a62c-1d24a988572e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:13.668049 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9befdb-ff1d-4eaf-a62c-1d24a988572e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:13.668322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9befdb-ff1d-4eaf-a62c-1d24a988572e None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:13.675409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9befdb-ff1d-4eaf-a62c-1d24a988572e None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:13.676897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9befdb-ff1d-4eaf-a62c-1d24a988572e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:13.678711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 99/203] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:13 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:13.679533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9befdb-ff1d-4eaf-a62c-1d24a988572e None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:13.760274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:13.772014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a5583-4dfc-4d02-bc89-89abf43a1376 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:13.773517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a5583-4dfc-4d02-bc89-89abf43a1376 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:13.816471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:13.854040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9befdb-ff1d-4eaf-a62c-1d24a988572e None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:13.907839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe1d0b-c2c8-4a44-85ba-7e2c7d9b522b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:13.908668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe1d0b-c2c8-4a44-85ba-7e2c7d9b522b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:13.909042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe1d0b-c2c8-4a44-85ba-7e2c7d9b522b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:13.909615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe1d0b-c2c8-4a44-85ba-7e2c7d9b522b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:13.910746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe1d0b-c2c8-4a44-85ba-7e2c7d9b522b None admin] RBAC: Adding query filter params (role.id=718dd530c67f4a1891e292e729330f51, scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859, user.id=a06a548fe9a74214ba8da3f9ece2d6fb) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:13.911060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe1d0b-c2c8-4a44-85ba-7e2c7d9b522b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:13.913128 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe1d0b-c2c8-4a44-85ba-7e2c7d9b522b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:13.918536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 105/204] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:13.919397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a5583-4dfc-4d02-bc89-89abf43a1376 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:13.928390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 100/205] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:13 2022] GET /identity/v3/role_assignments?user.id=a06a548fe9a74214ba8da3f9ece2d6fb&scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859&role.id=718dd530c67f4a1891e292e729330f51 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:13.931831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe1d0b-c2c8-4a44-85ba-7e2c7d9b522b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:14.069056 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:14.077343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a5583-4dfc-4d02-bc89-89abf43a1376 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:14.119879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2032c5e-07c3-4857-986b-0543b659b072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:14.120783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2032c5e-07c3-4857-986b-0543b659b072 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:14.121263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2032c5e-07c3-4857-986b-0543b659b072 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:14.121711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2032c5e-07c3-4857-986b-0543b659b072 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:14.122618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2032c5e-07c3-4857-986b-0543b659b072 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=53c2a41061214641a61babeb674afeae, user.id=3c7acd70496347beb8e901f1fbb8d4d0) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:14.123072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2032c5e-07c3-4857-986b-0543b659b072 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:14.124686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2032c5e-07c3-4857-986b-0543b659b072 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:14.131645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 106/206] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:13 2022] GET /identity/v3/role_assignments?user.id=3c7acd70496347beb8e901f1fbb8d4d0&scope.project.id=53c2a41061214641a61babeb674afeae&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:14.169558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe1d0b-c2c8-4a44-85ba-7e2c7d9b522b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:14.214254 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14e44c-0be9-4530-85d4-b46c8e7bf15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:14.214914 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14e44c-0be9-4530-85d4-b46c8e7bf15e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:14.215195 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14e44c-0be9-4530-85d4-b46c8e7bf15e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:14.215455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14e44c-0be9-4530-85d4-b46c8e7bf15e None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:14.216630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14e44c-0be9-4530-85d4-b46c8e7bf15e None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:14.218446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14e44c-0be9-4530-85d4-b46c8e7bf15e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:14.220509 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 101/207] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:13 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:14.226158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2032c5e-07c3-4857-986b-0543b659b072 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:14.344308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2032c5e-07c3-4857-986b-0543b659b072 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:14.383055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c6d00-ae98-4e5a-a14a-4384aff4bc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:14.384037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c6d00-ae98-4e5a-a14a-4384aff4bc83 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:14.384512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c6d00-ae98-4e5a-a14a-4384aff4bc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:14.384975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c6d00-ae98-4e5a-a14a-4384aff4bc83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:14.385986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c6d00-ae98-4e5a-a14a-4384aff4bc83 None admin] RBAC: Adding query filter params (role.id=89694ebde7824b8190bd5b15ece2b11e, scope.project.id=53c2a41061214641a61babeb674afeae, user.id=3c7acd70496347beb8e901f1fbb8d4d0) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:14.386495 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c6d00-ae98-4e5a-a14a-4384aff4bc83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:14.388158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c6d00-ae98-4e5a-a14a-4384aff4bc83 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:14.395387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 107/208] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:14 2022] GET /identity/v3/role_assignments?user.id=3c7acd70496347beb8e901f1fbb8d4d0&scope.project.id=53c2a41061214641a61babeb674afeae&role.id=89694ebde7824b8190bd5b15ece2b11e => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:14.997584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749cd0b-7fd3-4c9a-995e-ba03de612885 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:14.998131 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749cd0b-7fd3-4c9a-995e-ba03de612885 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:14.998131 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749cd0b-7fd3-4c9a-995e-ba03de612885 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:14.999480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 102/209] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:15.005507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32322cdb-c368-4aab-9a97-6537bb805f82 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:15.006371 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32322cdb-c368-4aab-9a97-6537bb805f82 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:15.006865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32322cdb-c368-4aab-9a97-6537bb805f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:15.028436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-32322cdb-c368-4aab-9a97-6537bb805f82 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:15.030183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-32322cdb-c368-4aab-9a97-6537bb805f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:15.050542 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:15.127711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 108/210] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:15.144514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb660d1-ac03-43c8-b5ac-ec558749b5d9 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:15.144848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb660d1-ac03-43c8-b5ac-ec558749b5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:15.145057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb660d1-ac03-43c8-b5ac-ec558749b5d9 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:15.146593 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 103/211] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:15.154370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14224000-63e0-4557-ab33-1db736fb337e None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:15.154830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14224000-63e0-4557-ab33-1db736fb337e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:15.155105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14224000-63e0-4557-ab33-1db736fb337e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:15.174905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-14224000-63e0-4557-ab33-1db736fb337e None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:15.176093 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-14224000-63e0-4557-ab33-1db736fb337e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:15.246400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 109/212] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:15.407314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb660d1-ac03-43c8-b5ac-ec558749b5d9 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:15.410726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:15.516887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb660d1-ac03-43c8-b5ac-ec558749b5d9 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:15.530786 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14224000-63e0-4557-ab33-1db736fb337e None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:15.559411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac38a8-b874-4162-b236-cdf09477479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:15.560057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac38a8-b874-4162-b236-cdf09477479d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:15.560337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac38a8-b874-4162-b236-cdf09477479d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:15.560611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac38a8-b874-4162-b236-cdf09477479d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:15.576681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac38a8-b874-4162-b236-cdf09477479d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-01ac38a8-b874-4162-b236-cdf09477479d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:15.583679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:15.584558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:15.585373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:15.585373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:15.585373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:15.585373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:15.585373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:15.585373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:15.585373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:15.585373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:15.585373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:15.585373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:15.585373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:15.585373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:15.592743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 104/213] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:15.594085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac38a8-b874-4162-b236-cdf09477479d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:15.630871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:15.651460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14224000-63e0-4557-ab33-1db736fb337e None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:15.662711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:15.690114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827b521-224a-403e-b548-c5d6de7959c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:15.690747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827b521-224a-403e-b548-c5d6de7959c7 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:15.691021 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827b521-224a-403e-b548-c5d6de7959c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:15.691287 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827b521-224a-403e-b548-c5d6de7959c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:15.693126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827b521-224a-403e-b548-c5d6de7959c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:15.694745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827b521-224a-403e-b548-c5d6de7959c7 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:15.696468 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 110/214] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:15.703902 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827b521-224a-403e-b548-c5d6de7959c7 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:15.719350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac38a8-b874-4162-b236-cdf09477479d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:15.756983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03805b9-c057-469d-b0a3-b2ed805946a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:15.757628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03805b9-c057-469d-b0a3-b2ed805946a2 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:15.757909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03805b9-c057-469d-b0a3-b2ed805946a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:15.758317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03805b9-c057-469d-b0a3-b2ed805946a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:15.758989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03805b9-c057-469d-b0a3-b2ed805946a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:15.759400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03805b9-c057-469d-b0a3-b2ed805946a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:15.760762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03805b9-c057-469d-b0a3-b2ed805946a2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:15.770591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 105/215] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:15.776559 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03805b9-c057-469d-b0a3-b2ed805946a2 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:15.813854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827b521-224a-403e-b548-c5d6de7959c7 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:15.871926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fededef-650e-4078-8497-d089a66bd5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:15.872544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fededef-650e-4078-8497-d089a66bd5de None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:15.873085 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fededef-650e-4078-8497-d089a66bd5de None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:15.873389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fededef-650e-4078-8497-d089a66bd5de None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:15.874950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fededef-650e-4078-8497-d089a66bd5de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:15.876404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fededef-650e-4078-8497-d089a66bd5de None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:15.909548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03805b9-c057-469d-b0a3-b2ed805946a2 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:15.913385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 111/216] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:24:15 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:24:15.953611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:15.955821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb4cd-2ffd-4b64-9bbb-e80c7314d7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:15.956797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb4cd-2ffd-4b64-9bbb-e80c7314d7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:15.957202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb4cd-2ffd-4b64-9bbb-e80c7314d7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:15.957577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb4cd-2ffd-4b64-9bbb-e80c7314d7c6 None admin] PATH_INFO: `/v3/users/899579935d8547ae9af7b7b2ac872f04` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:15.973920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fb4cd-2ffd-4b64-9bbb-e80c7314d7c6 None admin] RBAC: Authorizing `identity:get_user(user_id=899579935d8547ae9af7b7b2ac872f04)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:15.975472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fb4cd-2ffd-4b64-9bbb-e80c7314d7c6 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:15.977658 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 106/217] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:15 2022] GET /identity/v3/users/899579935d8547ae9af7b7b2ac872f04 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:15.986854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fededef-650e-4078-8497-d089a66bd5de None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:15.998158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:16.110189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fededef-650e-4078-8497-d089a66bd5de None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:16.154935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5fe167-4f50-4bf9-a051-f0d35d53c250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:16.155854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fe167-4f50-4bf9-a051-f0d35d53c250 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:16.156314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fe167-4f50-4bf9-a051-f0d35d53c250 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:16.156867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fe167-4f50-4bf9-a051-f0d35d53c250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:16.233344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5fe167-4f50-4bf9-a051-f0d35d53c250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:16.234836 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5fe167-4f50-4bf9-a051-f0d35d53c250 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:16.244271 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:16.362717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 112/218] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:16.368771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb4cd-2ffd-4b64-9bbb-e80c7314d7c6 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:16.474269 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28414ba2-503f-4b1d-8949-4a47365f9baa None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:16.474816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28414ba2-503f-4b1d-8949-4a47365f9baa None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:16.475248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28414ba2-503f-4b1d-8949-4a47365f9baa None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:16.478362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 113/219] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:24:16.486198 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc5bec-466d-42c1-a906-5bed88f43cfe None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:16.486797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc5bec-466d-42c1-a906-5bed88f43cfe None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:16.487339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc5bec-466d-42c1-a906-5bed88f43cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:16.489922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb4cd-2ffd-4b64-9bbb-e80c7314d7c6 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:16.518170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-c4fc5bec-466d-42c1-a906-5bed88f43cfe None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:16.519732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c4fc5bec-466d-42c1-a906-5bed88f43cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:16.555107 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277b603-05dc-4136-ba69-4848d75c4081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:16.555813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277b603-05dc-4136-ba69-4848d75c4081 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:16.556098 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277b603-05dc-4136-ba69-4848d75c4081 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:16.556376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277b603-05dc-4136-ba69-4848d75c4081 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:16.559281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277b603-05dc-4136-ba69-4848d75c4081 None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:16.561274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277b603-05dc-4136-ba69-4848d75c4081 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:16.567781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 107/220] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:16 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:16.570515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277b603-05dc-4136-ba69-4848d75c4081 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:16.654937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:16.675992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 114/221] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:16.699914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d0706-9e67-4139-8ede-7364011ed20a None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:16.700232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d0706-9e67-4139-8ede-7364011ed20a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:16.700468 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d0706-9e67-4139-8ede-7364011ed20a None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:16.701861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 115/222] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:16.716011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4f14e-36fd-4290-b6cc-51d3a62a3243 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:16.716300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4f14e-36fd-4290-b6cc-51d3a62a3243 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:16.716575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4f14e-36fd-4290-b6cc-51d3a62a3243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:16.740016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277b603-05dc-4136-ba69-4848d75c4081 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:16.751429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-02b4f14e-36fd-4290-b6cc-51d3a62a3243 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:16.752662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-02b4f14e-36fd-4290-b6cc-51d3a62a3243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:16.794737 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:16.815149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d484ed8-8017-4801-a7aa-e17c128e7e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:16.815778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484ed8-8017-4801-a7aa-e17c128e7e80 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:16.816058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484ed8-8017-4801-a7aa-e17c128e7e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:16.816332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d484ed8-8017-4801-a7aa-e17c128e7e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:16.817075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d484ed8-8017-4801-a7aa-e17c128e7e80 None admin] RBAC: Adding query filter params (role.id=4c80373ee4c34171a7f29e5ab6c09ce0, scope.project.id=53c2a41061214641a61babeb674afeae, user.id=899579935d8547ae9af7b7b2ac872f04) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:16.817407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d484ed8-8017-4801-a7aa-e17c128e7e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:16.818853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d484ed8-8017-4801-a7aa-e17c128e7e80 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:16.832313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 108/223] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:16 2022] GET /identity/v3/role_assignments?user.id=899579935d8547ae9af7b7b2ac872f04&scope.project.id=53c2a41061214641a61babeb674afeae&role.id=4c80373ee4c34171a7f29e5ab6c09ce0 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:16.857209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 116/224] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:16.995781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d484ed8-8017-4801-a7aa-e17c128e7e80 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:17.144043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d484ed8-8017-4801-a7aa-e17c128e7e80 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:17.163937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:17.191458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1bcee-e304-4148-8a7b-1e1adab3abda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:17.192103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1bcee-e304-4148-8a7b-1e1adab3abda None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:17.192386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1bcee-e304-4148-8a7b-1e1adab3abda None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:17.192664 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1bcee-e304-4148-8a7b-1e1adab3abda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:17.214297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1bcee-e304-4148-8a7b-1e1adab3abda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-26f1bcee-e304-4148-8a7b-1e1adab3abda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:17.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:17.220958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:17.221773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 109/225] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:24:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:17.227393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4f14e-36fd-4290-b6cc-51d3a62a3243 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:17.252043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1bcee-e304-4148-8a7b-1e1adab3abda None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:17.404572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4f14e-36fd-4290-b6cc-51d3a62a3243 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:17.447387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e77133-cc00-4cad-aa40-887e31b17904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:17.448018 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77133-cc00-4cad-aa40-887e31b17904 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:17.448295 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77133-cc00-4cad-aa40-887e31b17904 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:17.448567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77133-cc00-4cad-aa40-887e31b17904 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:17.449279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e77133-cc00-4cad-aa40-887e31b17904 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:17.449582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e77133-cc00-4cad-aa40-887e31b17904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:17.450980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e77133-cc00-4cad-aa40-887e31b17904 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:17.464666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1bcee-e304-4148-8a7b-1e1adab3abda None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:17.469922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 117/226] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:17.475657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e77133-cc00-4cad-aa40-887e31b17904 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:17.528079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93dbde1-a812-406c-bba5-b424c5f6638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:17.529176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93dbde1-a812-406c-bba5-b424c5f6638b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:17.529597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93dbde1-a812-406c-bba5-b424c5f6638b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:17.530006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93dbde1-a812-406c-bba5-b424c5f6638b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:17.539203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:17.548279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93dbde1-a812-406c-bba5-b424c5f6638b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-a93dbde1-a812-406c-bba5-b424c5f6638b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:17.554651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:17.555538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:17.556311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:17.556311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:17.556311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:17.556311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:17.556311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:17.556311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:17.556311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:17.556311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:17.556311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:17.556311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:17.556311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:17.556311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:17.558585 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 110/227] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:24:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:17.565010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93dbde1-a812-406c-bba5-b424c5f6638b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:17.625446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:17.663346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e77133-cc00-4cad-aa40-887e31b17904 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:17.700676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332db997-15ee-46aa-a0a2-4b1a188a4aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:17.701369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332db997-15ee-46aa-a0a2-4b1a188a4aea None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:17.701654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332db997-15ee-46aa-a0a2-4b1a188a4aea None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:17.701927 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332db997-15ee-46aa-a0a2-4b1a188a4aea None admin] PATH_INFO: `/v3/users/3c7acd70496347beb8e901f1fbb8d4d0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:17.703407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332db997-15ee-46aa-a0a2-4b1a188a4aea None admin] RBAC: Authorizing `identity:get_user(user_id=3c7acd70496347beb8e901f1fbb8d4d0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:17.705538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332db997-15ee-46aa-a0a2-4b1a188a4aea None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:17.707540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 118/228] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:17 2022] GET /identity/v3/users/3c7acd70496347beb8e901f1fbb8d4d0 => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:17.716455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332db997-15ee-46aa-a0a2-4b1a188a4aea None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:17.772623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93dbde1-a812-406c-bba5-b424c5f6638b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:17.858206 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d8eb4-3a5e-42be-92ba-383a944ffe51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:17.858738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d8eb4-3a5e-42be-92ba-383a944ffe51 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:17.859014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d8eb4-3a5e-42be-92ba-383a944ffe51 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:17.859342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d8eb4-3a5e-42be-92ba-383a944ffe51 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:17.860016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d8eb4-3a5e-42be-92ba-383a944ffe51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:17.860316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d8eb4-3a5e-42be-92ba-383a944ffe51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:17.863274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d8eb4-3a5e-42be-92ba-383a944ffe51 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:17.873474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:17.883679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 111/229] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:17.884606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d8eb4-3a5e-42be-92ba-383a944ffe51 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:17.887303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332db997-15ee-46aa-a0a2-4b1a188a4aea None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:17.953733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97744128-cfac-4a2c-b459-c63145246ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:17.954374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97744128-cfac-4a2c-b459-c63145246ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:17.954952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97744128-cfac-4a2c-b459-c63145246ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:17.955227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97744128-cfac-4a2c-b459-c63145246ea2 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:17.957152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97744128-cfac-4a2c-b459-c63145246ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:17.958598 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97744128-cfac-4a2c-b459-c63145246ea2 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:17.971602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 119/230] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:17 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:17.974303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97744128-cfac-4a2c-b459-c63145246ea2 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:18.085840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:18.099294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d8eb4-3a5e-42be-92ba-383a944ffe51 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:18.131224 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97744128-cfac-4a2c-b459-c63145246ea2 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:18.166794 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71497910-2aad-4916-83b2-dad129a75e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:18.167905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71497910-2aad-4916-83b2-dad129a75e93 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:18.168328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71497910-2aad-4916-83b2-dad129a75e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:18.168615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71497910-2aad-4916-83b2-dad129a75e93 None admin] PATH_INFO: `/v3/users/3c7acd70496347beb8e901f1fbb8d4d0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:18.174001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71497910-2aad-4916-83b2-dad129a75e93 None admin] RBAC: Authorizing `identity:get_user(user_id=3c7acd70496347beb8e901f1fbb8d4d0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:18.176455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71497910-2aad-4916-83b2-dad129a75e93 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:18.179021 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 112/231] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:17 2022] GET /identity/v3/users/3c7acd70496347beb8e901f1fbb8d4d0 => generated 283 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:18.184743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71497910-2aad-4916-83b2-dad129a75e93 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:18.189911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fc6d1-4b67-466a-9890-4add696c57e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:18.191801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fc6d1-4b67-466a-9890-4add696c57e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:18.197947 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fc6d1-4b67-466a-9890-4add696c57e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:18.198466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fc6d1-4b67-466a-9890-4add696c57e2 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae/users/3c7acd70496347beb8e901f1fbb8d4d0/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:18.200691 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fc6d1-4b67-466a-9890-4add696c57e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=53c2a41061214641a61babeb674afeae, user_id=3c7acd70496347beb8e901f1fbb8d4d0, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:18.202387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fc6d1-4b67-466a-9890-4add696c57e2 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:18.225911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 120/232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:24:17 2022] PUT /identity/v3/projects/53c2a41061214641a61babeb674afeae/users/3c7acd70496347beb8e901f1fbb8d4d0/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:18.320249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71497910-2aad-4916-83b2-dad129a75e93 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:18.360355 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523851ba-aeae-46b4-bc1f-dc0219daf1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:18.361422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523851ba-aeae-46b4-bc1f-dc0219daf1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:18.361923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523851ba-aeae-46b4-bc1f-dc0219daf1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:18.362384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523851ba-aeae-46b4-bc1f-dc0219daf1a3 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:18.363716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523851ba-aeae-46b4-bc1f-dc0219daf1a3 None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:18.365412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523851ba-aeae-46b4-bc1f-dc0219daf1a3 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:18.367681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 113/233] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:18 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:18.373274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fc6d1-4b67-466a-9890-4add696c57e2 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:18.410898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:18.498159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fc6d1-4b67-466a-9890-4add696c57e2 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:18.541930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae40cb-d155-41ef-9865-1a57eda5c6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:18.542608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae40cb-d155-41ef-9865-1a57eda5c6a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:18.542889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae40cb-d155-41ef-9865-1a57eda5c6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:18.543157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae40cb-d155-41ef-9865-1a57eda5c6a2 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae/users/3c7acd70496347beb8e901f1fbb8d4d0/roles/89694ebde7824b8190bd5b15ece2b11e` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:18.545347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae40cb-d155-41ef-9865-1a57eda5c6a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=53c2a41061214641a61babeb674afeae, user_id=3c7acd70496347beb8e901f1fbb8d4d0, role_id=89694ebde7824b8190bd5b15ece2b11e)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:18.546831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae40cb-d155-41ef-9865-1a57eda5c6a2 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:18.567489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 121/234] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:24:18 2022] PUT /identity/v3/projects/53c2a41061214641a61babeb674afeae/users/3c7acd70496347beb8e901f1fbb8d4d0/roles/89694ebde7824b8190bd5b15ece2b11e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:18.729566 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db9cfc-5864-4ce4-b407-34cc778c8c9b None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:18.730456 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db9cfc-5864-4ce4-b407-34cc778c8c9b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:18.730879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db9cfc-5864-4ce4-b407-34cc778c8c9b None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:18.732889 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 114/235] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:18.739212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4e61a-0a45-4baa-8bb9-2ac2cc099da8 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:18.739736 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4e61a-0a45-4baa-8bb9-2ac2cc099da8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:18.740152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4e61a-0a45-4baa-8bb9-2ac2cc099da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:18.765533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-8cc4e61a-0a45-4baa-8bb9-2ac2cc099da8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:18.767189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8cc4e61a-0a45-4baa-8bb9-2ac2cc099da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:18.856934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42108d8d-d51a-45c3-ad22-be21de9f6053 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:18.857997 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42108d8d-d51a-45c3-ad22-be21de9f6053 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:18.858279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42108d8d-d51a-45c3-ad22-be21de9f6053 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:18.859770 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 115/236] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:18.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483097df-499d-473c-ae8d-802ba992ce4c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:18.866930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483097df-499d-473c-ae8d-802ba992ce4c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:18.867437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483097df-499d-473c-ae8d-802ba992ce4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:18.887457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 122/237] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:18.888768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-483097df-499d-473c-ae8d-802ba992ce4c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:18.893745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-483097df-499d-473c-ae8d-802ba992ce4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:18.919566 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:18.965179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 116/238] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:19.200751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4e61a-0a45-4baa-8bb9-2ac2cc099da8 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:19.221120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:19.310899 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4e61a-0a45-4baa-8bb9-2ac2cc099da8 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:19.348421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483097df-499d-473c-ae8d-802ba992ce4c None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:19.356033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876b578-3b8a-4baf-99c2-1174951a8fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:19.357011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876b578-3b8a-4baf-99c2-1174951a8fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:19.357640 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876b578-3b8a-4baf-99c2-1174951a8fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:19.357921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876b578-3b8a-4baf-99c2-1174951a8fb4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:19.373887 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876b578-3b8a-4baf-99c2-1174951a8fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-1876b578-3b8a-4baf-99c2-1174951a8fb4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:19.379724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:19.380647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:19.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 123/239] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:19.389458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876b578-3b8a-4baf-99c2-1174951a8fb4 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:19.513483 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:19.552044 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876b578-3b8a-4baf-99c2-1174951a8fb4 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:19.562997 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483097df-499d-473c-ae8d-802ba992ce4c None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:19.623111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a8278-8c91-4482-8425-07a8962e334b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:19.624129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a8278-8c91-4482-8425-07a8962e334b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:19.624609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a8278-8c91-4482-8425-07a8962e334b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:19.625125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a8278-8c91-4482-8425-07a8962e334b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:19.626671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a8278-8c91-4482-8425-07a8962e334b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:19.628333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a8278-8c91-4482-8425-07a8962e334b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:19.630422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 117/240] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:19.636111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a8278-8c91-4482-8425-07a8962e334b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:19.645832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e703dd-c875-4fbc-b79d-02ba48f0521a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:19.646553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e703dd-c875-4fbc-b79d-02ba48f0521a None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:19.646992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e703dd-c875-4fbc-b79d-02ba48f0521a None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:19.647422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e703dd-c875-4fbc-b79d-02ba48f0521a None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:19.648111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e703dd-c875-4fbc-b79d-02ba48f0521a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:19.648416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e703dd-c875-4fbc-b79d-02ba48f0521a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:19.649810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e703dd-c875-4fbc-b79d-02ba48f0521a None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:19.660804 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 124/241] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:19.662370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7e432-22a4-405b-9b14-a044ad61c309 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:19.662859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7e432-22a4-405b-9b14-a044ad61c309 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:19.663404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7e432-22a4-405b-9b14-a044ad61c309 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:19.664874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 125/242] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:19.668000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7e432-22a4-405b-9b14-a044ad61c309 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:19.711073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:19.777378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a8278-8c91-4482-8425-07a8962e334b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:19.794187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7e432-22a4-405b-9b14-a044ad61c309 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:19.819921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676a0dd-d5f2-4297-9bf4-93e63412a5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:19.820796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676a0dd-d5f2-4297-9bf4-93e63412a5e6 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:19.821407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676a0dd-d5f2-4297-9bf4-93e63412a5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:19.822225 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676a0dd-d5f2-4297-9bf4-93e63412a5e6 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:19.823352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676a0dd-d5f2-4297-9bf4-93e63412a5e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:19.825518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676a0dd-d5f2-4297-9bf4-93e63412a5e6 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:19.838103 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bde05b-3f18-4640-ae8a-473a30e05dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:19.838972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bde05b-3f18-4640-ae8a-473a30e05dfe None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:19.839089 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bde05b-3f18-4640-ae8a-473a30e05dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:19.839303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bde05b-3f18-4640-ae8a-473a30e05dfe None admin] PATH_INFO: `/v3/users/3aa75836e72d443980eef361a42897a8` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:19.843016 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:19.854809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 118/243] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:24:19 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:24:19.856487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6de53-7b23-4a6e-b1ed-700fe64bbf09 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:19.857002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6de53-7b23-4a6e-b1ed-700fe64bbf09 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:19.857502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6de53-7b23-4a6e-b1ed-700fe64bbf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:19.876534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bde05b-3f18-4640-ae8a-473a30e05dfe None admin] RBAC: Authorizing `identity:get_user(user_id=3aa75836e72d443980eef361a42897a8)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:19.878157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bde05b-3f18-4640-ae8a-473a30e05dfe None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:19.880163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 126/244] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:19 2022] GET /identity/v3/users/3aa75836e72d443980eef361a42897a8 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:19.881036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-7fb6de53-7b23-4a6e-b1ed-700fe64bbf09 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:19.882534 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7fb6de53-7b23-4a6e-b1ed-700fe64bbf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:19.887175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bde05b-3f18-4640-ae8a-473a30e05dfe None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:20.046644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 119/245] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:20.123223 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bde05b-3f18-4640-ae8a-473a30e05dfe None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:20.164904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7201dda-7f03-4c81-9add-84135b00b177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:20.165607 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7201dda-7f03-4c81-9add-84135b00b177 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:20.165885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7201dda-7f03-4c81-9add-84135b00b177 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:20.166159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7201dda-7f03-4c81-9add-84135b00b177 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:20.239278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:20.247034 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7201dda-7f03-4c81-9add-84135b00b177 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:20.248495 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7201dda-7f03-4c81-9add-84135b00b177 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:20.352901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6de53-7b23-4a6e-b1ed-700fe64bbf09 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:20.374332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 127/246] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 08:24:20.379984 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:20.380218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7201dda-7f03-4c81-9add-84135b00b177 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:20.491126 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6de53-7b23-4a6e-b1ed-700fe64bbf09 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:20.494155 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:20.502010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7201dda-7f03-4c81-9add-84135b00b177 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:20.537034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdffbf3-5a3c-4247-b470-adbc05b87c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:20.537990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdffbf3-5a3c-4247-b470-adbc05b87c2a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:20.538348 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdffbf3-5a3c-4247-b470-adbc05b87c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:20.538698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdffbf3-5a3c-4247-b470-adbc05b87c2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:20.543013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b509e1-126a-49cf-92f4-b170883478e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:20.543671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b509e1-126a-49cf-92f4-b170883478e8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:20.543955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b509e1-126a-49cf-92f4-b170883478e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:20.544231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b509e1-126a-49cf-92f4-b170883478e8 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:20.545596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b509e1-126a-49cf-92f4-b170883478e8 None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:20.547252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b509e1-126a-49cf-92f4-b170883478e8 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:20.549846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 128/247] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:20 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:20.553798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdffbf3-5a3c-4247-b470-adbc05b87c2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:20.558044 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b509e1-126a-49cf-92f4-b170883478e8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-7bdffbf3-5a3c-4247-b470-adbc05b87c2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:20.558856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:20.559780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:20.560553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 120/248] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:20.572403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdffbf3-5a3c-4247-b470-adbc05b87c2a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:20.688008 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:20.699751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b509e1-126a-49cf-92f4-b170883478e8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:20.734409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdffbf3-5a3c-4247-b470-adbc05b87c2a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:20.764357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34cbb2-a22d-4a03-8e7c-c89318cf7686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:20.765019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34cbb2-a22d-4a03-8e7c-c89318cf7686 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:20.765336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34cbb2-a22d-4a03-8e7c-c89318cf7686 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:20.765609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34cbb2-a22d-4a03-8e7c-c89318cf7686 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:20.766352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34cbb2-a22d-4a03-8e7c-c89318cf7686 None admin] RBAC: Adding query filter params (role.id=718dd530c67f4a1891e292e729330f51, scope.project.id=53c2a41061214641a61babeb674afeae, user.id=3aa75836e72d443980eef361a42897a8) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:20.766667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34cbb2-a22d-4a03-8e7c-c89318cf7686 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:20.768151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34cbb2-a22d-4a03-8e7c-c89318cf7686 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:20.786074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 129/249] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:20 2022] GET /identity/v3/role_assignments?user.id=3aa75836e72d443980eef361a42897a8&scope.project.id=53c2a41061214641a61babeb674afeae&role.id=718dd530c67f4a1891e292e729330f51 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:20.802667 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814bc5a-0ba0-4db3-a3ca-5cb7cefc4480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:20.809743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814bc5a-0ba0-4db3-a3ca-5cb7cefc4480 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:20.810319 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814bc5a-0ba0-4db3-a3ca-5cb7cefc4480 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:20.810772 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814bc5a-0ba0-4db3-a3ca-5cb7cefc4480 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:20.811691 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814bc5a-0ba0-4db3-a3ca-5cb7cefc4480 None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:20.812253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814bc5a-0ba0-4db3-a3ca-5cb7cefc4480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:20.813960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814bc5a-0ba0-4db3-a3ca-5cb7cefc4480 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:20.824732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 121/250] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:20.831060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34cbb2-a22d-4a03-8e7c-c89318cf7686 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:20.921755 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:20.938774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46310d0-7f8f-4f7b-903c-40af425e6388 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:20.939411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46310d0-7f8f-4f7b-903c-40af425e6388 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:20.939923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46310d0-7f8f-4f7b-903c-40af425e6388 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:20.943990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 122/251] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:20.950561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7c38e-4531-4015-8661-5e2920e60e43 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:20.951167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7c38e-4531-4015-8661-5e2920e60e43 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:20.951639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7c38e-4531-4015-8661-5e2920e60e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:20.979535 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34cbb2-a22d-4a03-8e7c-c89318cf7686 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:20.980595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-3fb7c38e-4531-4015-8661-5e2920e60e43 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:20.982045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3fb7c38e-4531-4015-8661-5e2920e60e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:21.056291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85c6b2-241d-40f1-a190-2df48f7de2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:21.057984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85c6b2-241d-40f1-a190-2df48f7de2ee None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:21.059489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85c6b2-241d-40f1-a190-2df48f7de2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:21.060010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85c6b2-241d-40f1-a190-2df48f7de2ee None admin] PATH_INFO: `/v3/users/899579935d8547ae9af7b7b2ac872f04` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:21.062560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85c6b2-241d-40f1-a190-2df48f7de2ee None admin] RBAC: Authorizing `identity:get_user(user_id=899579935d8547ae9af7b7b2ac872f04)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:21.069982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85c6b2-241d-40f1-a190-2df48f7de2ee None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:21.072278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 130/252] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:20 2022] GET /identity/v3/users/899579935d8547ae9af7b7b2ac872f04 => generated 297 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:21.079336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85c6b2-241d-40f1-a190-2df48f7de2ee None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:21.143033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 123/253] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:21.223392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:21.232422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85c6b2-241d-40f1-a190-2df48f7de2ee None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:21.271387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ffa5e-86aa-4298-993b-030a21a608fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:21.272423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ffa5e-86aa-4298-993b-030a21a608fd None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:21.272907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ffa5e-86aa-4298-993b-030a21a608fd None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:21.273423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ffa5e-86aa-4298-993b-030a21a608fd None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:21.274842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ffa5e-86aa-4298-993b-030a21a608fd None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:21.276809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ffa5e-86aa-4298-993b-030a21a608fd None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:21.279116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 131/254] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:21 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:21.284867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7c38e-4531-4015-8661-5e2920e60e43 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:21.392812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:21.440875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7c38e-4531-4015-8661-5e2920e60e43 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:21.492378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639ca8e-230e-4c8f-96a0-221a394021d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:21.492990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639ca8e-230e-4c8f-96a0-221a394021d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:21.493296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639ca8e-230e-4c8f-96a0-221a394021d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:21.493614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639ca8e-230e-4c8f-96a0-221a394021d2 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae/users/899579935d8547ae9af7b7b2ac872f04/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:21.495886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639ca8e-230e-4c8f-96a0-221a394021d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=53c2a41061214641a61babeb674afeae, user_id=899579935d8547ae9af7b7b2ac872f04, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:21.497361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639ca8e-230e-4c8f-96a0-221a394021d2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:21.503239 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f6602-4cac-4cd2-9fbe-894a7e9127f4 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:21.503720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f6602-4cac-4cd2-9fbe-894a7e9127f4 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:21.503792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f6602-4cac-4cd2-9fbe-894a7e9127f4 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:21.505290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 132/255] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:24:21.511074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f99e-f281-4606-ab6c-a798801defea None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:21.511434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f99e-f281-4606-ab6c-a798801defea None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:21.511745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f99e-f281-4606-ab6c-a798801defea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:21.536285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 124/256] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:24:21 2022] PUT /identity/v3/projects/53c2a41061214641a61babeb674afeae/users/899579935d8547ae9af7b7b2ac872f04/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:21.547362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-3c76f99e-f281-4606-ab6c-a798801defea None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:21.548641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3c76f99e-f281-4606-ab6c-a798801defea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:21.643915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639ca8e-230e-4c8f-96a0-221a394021d2 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:21.644921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 133/257] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:21.826073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639ca8e-230e-4c8f-96a0-221a394021d2 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:21.874144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:21.883955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdf2bd-4b8e-48cb-b19e-1f6ee5affd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:21.884626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdf2bd-4b8e-48cb-b19e-1f6ee5affd57 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:21.884946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdf2bd-4b8e-48cb-b19e-1f6ee5affd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:21.885253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdf2bd-4b8e-48cb-b19e-1f6ee5affd57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:21.901393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bdf2bd-4b8e-48cb-b19e-1f6ee5affd57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-28bdf2bd-4b8e-48cb-b19e-1f6ee5affd57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:21.906365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:21.907230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:21.908077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 125/258] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:24:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:21.916232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76f99e-f281-4606-ab6c-a798801defea None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:22.026956 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdf2bd-4b8e-48cb-b19e-1f6ee5affd57 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:22.039856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:22.052057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76f99e-f281-4606-ab6c-a798801defea None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:22.093744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56be7af-1bdc-491d-9a8d-f131cc800012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:22.094966 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56be7af-1bdc-491d-9a8d-f131cc800012 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:22.095547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56be7af-1bdc-491d-9a8d-f131cc800012 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:22.096121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56be7af-1bdc-491d-9a8d-f131cc800012 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:22.097012 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56be7af-1bdc-491d-9a8d-f131cc800012 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:22.097983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56be7af-1bdc-491d-9a8d-f131cc800012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:22.099751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56be7af-1bdc-491d-9a8d-f131cc800012 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:22.115823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 134/259] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:22.126498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56be7af-1bdc-491d-9a8d-f131cc800012 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:22.259101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdf2bd-4b8e-48cb-b19e-1f6ee5affd57 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:22.316876 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56be7af-1bdc-491d-9a8d-f131cc800012 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:22.337290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b17756-b87e-4def-bd47-9af51b346ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:22.338167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17756-b87e-4def-bd47-9af51b346ada None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:22.338346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17756-b87e-4def-bd47-9af51b346ada None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:22.338587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17756-b87e-4def-bd47-9af51b346ada None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:22.358380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc4132-ac6a-4c59-8874-3db67f5c052f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:22.359842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc4132-ac6a-4c59-8874-3db67f5c052f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:22.360330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc4132-ac6a-4c59-8874-3db67f5c052f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:22.360873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc4132-ac6a-4c59-8874-3db67f5c052f None admin] PATH_INFO: `/v3/users/3c7acd70496347beb8e901f1fbb8d4d0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:22.366151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc4132-ac6a-4c59-8874-3db67f5c052f None admin] RBAC: Authorizing `identity:get_user(user_id=3c7acd70496347beb8e901f1fbb8d4d0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:22.366973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b17756-b87e-4def-bd47-9af51b346ada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:22.371370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc4132-ac6a-4c59-8874-3db67f5c052f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:22.373317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 135/260] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:22 2022] GET /identity/v3/users/3c7acd70496347beb8e901f1fbb8d4d0 => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-f4b17756-b87e-4def-bd47-9af51b346ada None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:22.380338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:22.381283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:22.382143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:22.382143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:22.382143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:22.382143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:22.382143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:22.382143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:22.382143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:22.382143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:22.382143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:22.382143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:22.382143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:22.382143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:22.384449 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 126/261] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:24:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:22.386957 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc4132-ac6a-4c59-8874-3db67f5c052f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:22.393473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b17756-b87e-4def-bd47-9af51b346ada None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:22.429079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:22.520015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:22.540177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc4132-ac6a-4c59-8874-3db67f5c052f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:22.586397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fc103-485d-4864-a0f3-50164f586633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:22.587637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fc103-485d-4864-a0f3-50164f586633 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:22.589960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fc103-485d-4864-a0f3-50164f586633 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:22.590480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fc103-485d-4864-a0f3-50164f586633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:22.655450 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b17756-b87e-4def-bd47-9af51b346ada None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:22.673168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215fc103-485d-4864-a0f3-50164f586633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:22.675138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215fc103-485d-4864-a0f3-50164f586633 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:22.705537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:22.719239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f78e2d-f9be-474a-adbf-67a786ea8e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:22.719941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f78e2d-f9be-474a-adbf-67a786ea8e4c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:22.720832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f78e2d-f9be-474a-adbf-67a786ea8e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:22.721135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f78e2d-f9be-474a-adbf-67a786ea8e4c None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:22.721811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f78e2d-f9be-474a-adbf-67a786ea8e4c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:22.722118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f78e2d-f9be-474a-adbf-67a786ea8e4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:22.728366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f78e2d-f9be-474a-adbf-67a786ea8e4c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:22.752169 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 127/262] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:22.754142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f78e2d-f9be-474a-adbf-67a786ea8e4c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:22.891825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 136/263] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:22.903664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fc103-485d-4864-a0f3-50164f586633 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:22.938445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f78e2d-f9be-474a-adbf-67a786ea8e4c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:22.959190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:23.005119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c1a52-7bd9-4452-accb-49e1dab15384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:23.005119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c1a52-7bd9-4452-accb-49e1dab15384 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:23.005119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c1a52-7bd9-4452-accb-49e1dab15384 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:23.005119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c1a52-7bd9-4452-accb-49e1dab15384 None admin] PATH_INFO: `/v3/users/3c7acd70496347beb8e901f1fbb8d4d0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:23.005522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c1a52-7bd9-4452-accb-49e1dab15384 None admin] RBAC: Authorizing `identity:get_user(user_id=3c7acd70496347beb8e901f1fbb8d4d0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:23.007833 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c1a52-7bd9-4452-accb-49e1dab15384 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:23.009781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 128/264] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:22 2022] GET /identity/v3/users/3c7acd70496347beb8e901f1fbb8d4d0 => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:23.022248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c1a52-7bd9-4452-accb-49e1dab15384 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:23.130682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fc103-485d-4864-a0f3-50164f586633 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:23.170404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:23.187124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780f90c-3f2a-4834-974d-cddb06696225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:23.187882 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780f90c-3f2a-4834-974d-cddb06696225 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:23.188165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780f90c-3f2a-4834-974d-cddb06696225 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:23.188436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780f90c-3f2a-4834-974d-cddb06696225 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:23.194340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780f90c-3f2a-4834-974d-cddb06696225 None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:23.196756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780f90c-3f2a-4834-974d-cddb06696225 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:23.199055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 137/265] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:22 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:23.206106 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780f90c-3f2a-4834-974d-cddb06696225 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:23.260857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c1a52-7bd9-4452-accb-49e1dab15384 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:23.324544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c297b-e11a-4fb9-97fe-f76e8cb8eef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:23.325589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c297b-e11a-4fb9-97fe-f76e8cb8eef9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:23.326072 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c297b-e11a-4fb9-97fe-f76e8cb8eef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:23.326539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c297b-e11a-4fb9-97fe-f76e8cb8eef9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:23.403124 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:23.447428 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c297b-e11a-4fb9-97fe-f76e8cb8eef9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:23.449251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c297b-e11a-4fb9-97fe-f76e8cb8eef9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:23.491827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780f90c-3f2a-4834-974d-cddb06696225 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:23.573826 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb717c-272b-48a3-b091-34ac58974eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:23.574486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb717c-272b-48a3-b091-34ac58974eda None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:23.574700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb717c-272b-48a3-b091-34ac58974eda None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:23.574968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb717c-272b-48a3-b091-34ac58974eda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:23.575704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb717c-272b-48a3-b091-34ac58974eda None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=53c2a41061214641a61babeb674afeae, user.id=3c7acd70496347beb8e901f1fbb8d4d0) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:23.576010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb717c-272b-48a3-b091-34ac58974eda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:23.577842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb717c-272b-48a3-b091-34ac58974eda None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:23.593630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 138/266] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:23 2022] GET /identity/v3/role_assignments?user.id=3c7acd70496347beb8e901f1fbb8d4d0&scope.project.id=53c2a41061214641a61babeb674afeae&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 608 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:23.595240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db26a0-b2b0-4db9-8a84-9002bd825381 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:23.595358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db26a0-b2b0-4db9-8a84-9002bd825381 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:23.598936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db26a0-b2b0-4db9-8a84-9002bd825381 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:23.604990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 139/267] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:23 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:23.618146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483d46d-01f6-4062-8532-b32bef11ad97 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:23.618450 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483d46d-01f6-4062-8532-b32bef11ad97 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:23.618705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483d46d-01f6-4062-8532-b32bef11ad97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:23.660757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f483d46d-01f6-4062-8532-b32bef11ad97 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:23.662050 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f483d46d-01f6-4062-8532-b32bef11ad97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:23.719421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 129/268] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:23.727074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c297b-e11a-4fb9-97fe-f76e8cb8eef9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:23.837133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:23.862363 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 140/269] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:23.863445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bcb31-b03f-4238-9dc3-7b19f1146637 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:23.863730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bcb31-b03f-4238-9dc3-7b19f1146637 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:23.864152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bcb31-b03f-4238-9dc3-7b19f1146637 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:23.865744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 141/270] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:23.873588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc45ef-ec7e-4a4f-8288-fb4b35ae3b4a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:23.874025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc45ef-ec7e-4a4f-8288-fb4b35ae3b4a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:23.874186 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc45ef-ec7e-4a4f-8288-fb4b35ae3b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:23.910109 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:23.914283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-95cc45ef-ec7e-4a4f-8288-fb4b35ae3b4a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:23.915518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-95cc45ef-ec7e-4a4f-8288-fb4b35ae3b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:23.962426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c297b-e11a-4fb9-97fe-f76e8cb8eef9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:24.031555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227229f-5191-476f-84c9-346e8d6f652d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:24.033271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227229f-5191-476f-84c9-346e8d6f652d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:24.034978 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227229f-5191-476f-84c9-346e8d6f652d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:24.035803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227229f-5191-476f-84c9-346e8d6f652d None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:24.038154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 142/271] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:24.040857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227229f-5191-476f-84c9-346e8d6f652d None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:24.042574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227229f-5191-476f-84c9-346e8d6f652d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:24.044760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 130/272] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:23 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:24.051025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc45ef-ec7e-4a4f-8288-fb4b35ae3b4a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:24.177879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc45ef-ec7e-4a4f-8288-fb4b35ae3b4a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:24.200479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:24.227839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d15c99-baab-47a9-8ae4-1344ce8b7b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:24.228459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d15c99-baab-47a9-8ae4-1344ce8b7b29 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:24.228739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d15c99-baab-47a9-8ae4-1344ce8b7b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:24.229014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d15c99-baab-47a9-8ae4-1344ce8b7b29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:24.229788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d15c99-baab-47a9-8ae4-1344ce8b7b29 None admin] RBAC: Adding query filter params (role.id=89694ebde7824b8190bd5b15ece2b11e, scope.project.id=53c2a41061214641a61babeb674afeae, user.id=3c7acd70496347beb8e901f1fbb8d4d0) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:24.230097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d15c99-baab-47a9-8ae4-1344ce8b7b29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:24.231563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d15c99-baab-47a9-8ae4-1344ce8b7b29 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:24.242534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 143/273] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:24 2022] GET /identity/v3/role_assignments?user.id=3c7acd70496347beb8e901f1fbb8d4d0&scope.project.id=53c2a41061214641a61babeb674afeae&role.id=89694ebde7824b8190bd5b15ece2b11e => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:24.265051 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227229f-5191-476f-84c9-346e8d6f652d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:24.347438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d15c99-baab-47a9-8ae4-1344ce8b7b29 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:24.470950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227229f-5191-476f-84c9-346e8d6f652d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:24.495515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d15c99-baab-47a9-8ae4-1344ce8b7b29 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:24.522214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57aa7fd-cddb-45b1-b597-a4b5d19340f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:24.523247 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57aa7fd-cddb-45b1-b597-a4b5d19340f8 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:24.523746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57aa7fd-cddb-45b1-b597-a4b5d19340f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:24.524221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57aa7fd-cddb-45b1-b597-a4b5d19340f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:24.539493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57aa7fd-cddb-45b1-b597-a4b5d19340f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-c57aa7fd-cddb-45b1-b597-a4b5d19340f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:24.544861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:24.545825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:24.546620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:24.546620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:24.546620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:24.546620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:24.546620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:24.546620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:24.546620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:24.546620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:24.546620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:24.546620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:24.546620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:24.546620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:24.550000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 131/274] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:24.550125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72d825-8c1c-4871-9523-617368e148f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:24.550672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72d825-8c1c-4871-9523-617368e148f6 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:24.550955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72d825-8c1c-4871-9523-617368e148f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:24.551232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72d825-8c1c-4871-9523-617368e148f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:24.553325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72d825-8c1c-4871-9523-617368e148f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:24.553991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72d825-8c1c-4871-9523-617368e148f6 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:24.556153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 144/275] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:24.556674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57aa7fd-cddb-45b1-b597-a4b5d19340f8 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:24.563286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72d825-8c1c-4871-9523-617368e148f6 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:24.569270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:24.638461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:24.706285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57aa7fd-cddb-45b1-b597-a4b5d19340f8 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:24.723063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72d825-8c1c-4871-9523-617368e148f6 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:24.751922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91d9a9-3113-4f51-8424-6106f8a5d726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:24.752547 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91d9a9-3113-4f51-8424-6106f8a5d726 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:24.752826 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91d9a9-3113-4f51-8424-6106f8a5d726 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:24.753131 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91d9a9-3113-4f51-8424-6106f8a5d726 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:24.753830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91d9a9-3113-4f51-8424-6106f8a5d726 None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:24.754132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91d9a9-3113-4f51-8424-6106f8a5d726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:24.755536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91d9a9-3113-4f51-8424-6106f8a5d726 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:24.769334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 132/276] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:24.772162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bee84-5a71-4d18-a089-b55483495d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:24.773413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bee84-5a71-4d18-a089-b55483495d2d None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:24.773925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bee84-5a71-4d18-a089-b55483495d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:24.774421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bee84-5a71-4d18-a089-b55483495d2d None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:24.774669 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91d9a9-3113-4f51-8424-6106f8a5d726 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:24.776425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37bee84-5a71-4d18-a089-b55483495d2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:24.778596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37bee84-5a71-4d18-a089-b55483495d2d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:24.814208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 145/277] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:24:24 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:24:24.815918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95babc01-4820-442a-ba37-106bce9a1961 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:24.816372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95babc01-4820-442a-ba37-106bce9a1961 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:24.816658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95babc01-4820-442a-ba37-106bce9a1961 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:24.818718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 146/278] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:24.823008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d9474-8e61-4712-9a32-ae37eae0fce7 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:24.823008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d9474-8e61-4712-9a32-ae37eae0fce7 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:24.823008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d9474-8e61-4712-9a32-ae37eae0fce7 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:24.823374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 147/279] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:24.824974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2062c4-afb9-4346-846d-f2fe90bd2160 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:24.825759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2062c4-afb9-4346-846d-f2fe90bd2160 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:24.826123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2062c4-afb9-4346-846d-f2fe90bd2160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:24.851996 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-fa2062c4-afb9-4346-846d-f2fe90bd2160 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:24.853310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-fa2062c4-afb9-4346-846d-f2fe90bd2160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:24.881354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:24.927028 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91d9a9-3113-4f51-8424-6106f8a5d726 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:24.948614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 148/280] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:24.961331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc076a-3d55-4048-b27a-b19fdbc99244 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:24.961331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc076a-3d55-4048-b27a-b19fdbc99244 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:24.961331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc076a-3d55-4048-b27a-b19fdbc99244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:24.968745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64743eb9-2290-4767-a282-963c282d78b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:24.969437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64743eb9-2290-4767-a282-963c282d78b1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:24.969727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64743eb9-2290-4767-a282-963c282d78b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:24.970001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64743eb9-2290-4767-a282-963c282d78b1 None admin] PATH_INFO: `/v3/users/566a8ff23a9940c392826071f5ff7295` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:24.981498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f0cc076a-3d55-4048-b27a-b19fdbc99244 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:24.983137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f0cc076a-3d55-4048-b27a-b19fdbc99244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:24.987980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64743eb9-2290-4767-a282-963c282d78b1 None admin] RBAC: Authorizing `identity:get_user(user_id=566a8ff23a9940c392826071f5ff7295)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:24.989649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64743eb9-2290-4767-a282-963c282d78b1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:24.991879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 133/281] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:24 2022] GET /identity/v3/users/566a8ff23a9940c392826071f5ff7295 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:24.997716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64743eb9-2290-4767-a282-963c282d78b1 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:25.025216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:25.085312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 149/282] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:25.138429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64743eb9-2290-4767-a282-963c282d78b1 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:25.168674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:25.181478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa9743-20b6-4281-adb8-5dfff667a8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:25.182023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa9743-20b6-4281-adb8-5dfff667a8fd None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:25.182302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa9743-20b6-4281-adb8-5dfff667a8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:25.182574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa9743-20b6-4281-adb8-5dfff667a8fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:25.183305 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa9743-20b6-4281-adb8-5dfff667a8fd None admin] RBAC: Adding query filter params (role.id=4c80373ee4c34171a7f29e5ab6c09ce0, user.id=566a8ff23a9940c392826071f5ff7295, scope.system=all) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:25.183683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa9743-20b6-4281-adb8-5dfff667a8fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:25.187657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa9743-20b6-4281-adb8-5dfff667a8fd None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:25.194785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 134/283] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Sep 28 08:24:24 2022] GET /identity/v3/role_assignments?user.id=566a8ff23a9940c392826071f5ff7295&scope.system=all&role.id=4c80373ee4c34171a7f29e5ab6c09ce0 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:25.235172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc076a-3d55-4048-b27a-b19fdbc99244 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:25.365828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc076a-3d55-4048-b27a-b19fdbc99244 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:25.407076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71e4bc-6ded-4d8f-857b-ba7ec37108c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:25.408334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71e4bc-6ded-4d8f-857b-ba7ec37108c7 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:25.408667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71e4bc-6ded-4d8f-857b-ba7ec37108c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:25.408945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71e4bc-6ded-4d8f-857b-ba7ec37108c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:25.418556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa9743-20b6-4281-adb8-5dfff667a8fd None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:25.424799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71e4bc-6ded-4d8f-857b-ba7ec37108c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-4a71e4bc-6ded-4d8f-857b-ba7ec37108c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:25.430828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:25.431818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:25.432614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 150/284] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:25.438434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71e4bc-6ded-4d8f-857b-ba7ec37108c7 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:25.475143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:25.575557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa9743-20b6-4281-adb8-5dfff667a8fd None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:25.610210 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71e4bc-6ded-4d8f-857b-ba7ec37108c7 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:25.613804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc2c7d-3f8c-4570-850c-968c9e9f56a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:25.614627 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc2c7d-3f8c-4570-850c-968c9e9f56a9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:25.615119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc2c7d-3f8c-4570-850c-968c9e9f56a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:25.615589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc2c7d-3f8c-4570-850c-968c9e9f56a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:25.635138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc2c7d-3f8c-4570-850c-968c9e9f56a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-cecc2c7d-3f8c-4570-850c-968c9e9f56a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:25.649117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:25.650191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:25.651415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:25.651415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:25.651415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:25.651415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:25.651415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:25.651415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:25.651415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:25.651415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:25.651415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:25.651415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:25.651415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:25.651415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:25.654222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 135/285] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:25.655913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a8c1d-96dc-479d-853c-321e9f069924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:25.656657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a8c1d-96dc-479d-853c-321e9f069924 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:25.657036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a8c1d-96dc-479d-853c-321e9f069924 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:25.657439 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a8c1d-96dc-479d-853c-321e9f069924 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:25.658839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a8c1d-96dc-479d-853c-321e9f069924 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:25.659332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a8c1d-96dc-479d-853c-321e9f069924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:25.660457 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc2c7d-3f8c-4570-850c-968c9e9f56a9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:25.660717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a8c1d-96dc-479d-853c-321e9f069924 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:25.678490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 151/286] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:25.690014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a8c1d-96dc-479d-853c-321e9f069924 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:25.733194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:25.816313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc2c7d-3f8c-4570-850c-968c9e9f56a9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:25.857573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce3fad-b7ab-46c0-92e3-f0dec892ddba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:25.858190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce3fad-b7ab-46c0-92e3-f0dec892ddba None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:25.858471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce3fad-b7ab-46c0-92e3-f0dec892ddba None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:25.858749 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce3fad-b7ab-46c0-92e3-f0dec892ddba None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:25.859413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce3fad-b7ab-46c0-92e3-f0dec892ddba None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:25.859716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce3fad-b7ab-46c0-92e3-f0dec892ddba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:25.862703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce3fad-b7ab-46c0-92e3-f0dec892ddba None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:25.866922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:25.871824 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 136/287] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:25.878865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce3fad-b7ab-46c0-92e3-f0dec892ddba None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:25.904199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a8c1d-96dc-479d-853c-321e9f069924 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:25.979943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a1cff-7d7d-40f5-bb66-964b79004c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:25.980963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a1cff-7d7d-40f5-bb66-964b79004c90 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:25.981462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a1cff-7d7d-40f5-bb66-964b79004c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:25.981929 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a1cff-7d7d-40f5-bb66-964b79004c90 None admin] PATH_INFO: `/v3/users/3aa75836e72d443980eef361a42897a8` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:25.985890 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a1cff-7d7d-40f5-bb66-964b79004c90 None admin] RBAC: Authorizing `identity:get_user(user_id=3aa75836e72d443980eef361a42897a8)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:25.987638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a1cff-7d7d-40f5-bb66-964b79004c90 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:25.993118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 152/288] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:25 2022] GET /identity/v3/users/3aa75836e72d443980eef361a42897a8 => generated 297 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:25.995398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a1cff-7d7d-40f5-bb66-964b79004c90 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:26.063680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce3fad-b7ab-46c0-92e3-f0dec892ddba None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:26.082042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:26.130236 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470db0d-9a1e-4b01-96db-4a3295f5a57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:26.130891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470db0d-9a1e-4b01-96db-4a3295f5a57d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:26.131178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470db0d-9a1e-4b01-96db-4a3295f5a57d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:26.131459 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470db0d-9a1e-4b01-96db-4a3295f5a57d None admin] PATH_INFO: `/v3/users/899579935d8547ae9af7b7b2ac872f04` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:26.133394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470db0d-9a1e-4b01-96db-4a3295f5a57d None admin] RBAC: Authorizing `identity:get_user(user_id=899579935d8547ae9af7b7b2ac872f04)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:26.134900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470db0d-9a1e-4b01-96db-4a3295f5a57d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:26.136579 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 137/289] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:25 2022] GET /identity/v3/users/899579935d8547ae9af7b7b2ac872f04 => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:26.143420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470db0d-9a1e-4b01-96db-4a3295f5a57d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:26.203021 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a1cff-7d7d-40f5-bb66-964b79004c90 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:26.247275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b9e7b-30a8-4a2b-a923-425964e534df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:26.248343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b9e7b-30a8-4a2b-a923-425964e534df None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:26.248843 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b9e7b-30a8-4a2b-a923-425964e534df None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:26.249368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b9e7b-30a8-4a2b-a923-425964e534df None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:26.250866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b9e7b-30a8-4a2b-a923-425964e534df None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:26.252789 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b9e7b-30a8-4a2b-a923-425964e534df None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:26.255852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 153/290] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:25 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:26.261207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b9e7b-30a8-4a2b-a923-425964e534df None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:26.289163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:26.294448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470db0d-9a1e-4b01-96db-4a3295f5a57d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:26.331470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e347d3-7531-4b77-908b-4064882c8d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:26.332093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e347d3-7531-4b77-908b-4064882c8d00 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:26.332373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e347d3-7531-4b77-908b-4064882c8d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:26.332643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e347d3-7531-4b77-908b-4064882c8d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:26.388097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b9e7b-30a8-4a2b-a923-425964e534df None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:26.447935 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:26.468730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ae741-49e6-49c1-97b9-b7dccd920530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:26.469816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ae741-49e6-49c1-97b9-b7dccd920530 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:26.470892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ae741-49e6-49c1-97b9-b7dccd920530 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:26.471370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ae741-49e6-49c1-97b9-b7dccd920530 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae/users/3aa75836e72d443980eef361a42897a8/roles/718dd530c67f4a1891e292e729330f51` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:26.473640 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ae741-49e6-49c1-97b9-b7dccd920530 None admin] RBAC: Authorizing `identity:create_grant(project_id=53c2a41061214641a61babeb674afeae, user_id=3aa75836e72d443980eef361a42897a8, role_id=718dd530c67f4a1891e292e729330f51)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:26.475337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ae741-49e6-49c1-97b9-b7dccd920530 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:26.491469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e347d3-7531-4b77-908b-4064882c8d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:26.495589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e347d3-7531-4b77-908b-4064882c8d00 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:26.500915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 154/291] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:24:26 2022] PUT /identity/v3/projects/53c2a41061214641a61babeb674afeae/users/3aa75836e72d443980eef361a42897a8/roles/718dd530c67f4a1891e292e729330f51 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:26.642936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 138/292] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:26.652293 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ae741-49e6-49c1-97b9-b7dccd920530 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:26.766700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:26.800702 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ae741-49e6-49c1-97b9-b7dccd920530 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:26.850797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5921c-af8d-4c62-9b34-353331584669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:26.851442 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5921c-af8d-4c62-9b34-353331584669 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:26.851721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5921c-af8d-4c62-9b34-353331584669 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:26.851991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5921c-af8d-4c62-9b34-353331584669 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:26.853160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5921c-af8d-4c62-9b34-353331584669 None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:26.854698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5921c-af8d-4c62-9b34-353331584669 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:26.857664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 155/293] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:26 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:26.862755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e347d3-7531-4b77-908b-4064882c8d00 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:26.953952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:26.978868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e347d3-7531-4b77-908b-4064882c8d00 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:27.030263 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629d715-8aca-450b-9c04-7bcba9492b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:27.030793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629d715-8aca-450b-9c04-7bcba9492b76 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:27.031071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629d715-8aca-450b-9c04-7bcba9492b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:27.031341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629d715-8aca-450b-9c04-7bcba9492b76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:27.032066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629d715-8aca-450b-9c04-7bcba9492b76 None admin] RBAC: Adding query filter params (role.id=4c80373ee4c34171a7f29e5ab6c09ce0, scope.project.id=53c2a41061214641a61babeb674afeae, user.id=899579935d8547ae9af7b7b2ac872f04) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:27.032373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629d715-8aca-450b-9c04-7bcba9492b76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:27.033859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629d715-8aca-450b-9c04-7bcba9492b76 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:27.044283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 139/294] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:26 2022] GET /identity/v3/role_assignments?user.id=899579935d8547ae9af7b7b2ac872f04&scope.project.id=53c2a41061214641a61babeb674afeae&role.id=4c80373ee4c34171a7f29e5ab6c09ce0 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:27.684200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74dec3-512c-4371-ab2e-dbb659eda184 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:27.684615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74dec3-512c-4371-ab2e-dbb659eda184 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:27.684757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74dec3-512c-4371-ab2e-dbb659eda184 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:27.686457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 156/295] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:27.691821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33465f-a70b-4b46-8225-a611b9c14929 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:27.692268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33465f-a70b-4b46-8225-a611b9c14929 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:27.692448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33465f-a70b-4b46-8225-a611b9c14929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:27.711814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-bc33465f-a70b-4b46-8225-a611b9c14929 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:27.713032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-bc33465f-a70b-4b46-8225-a611b9c14929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:27.766807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581cd94-8c6a-435b-b478-52b6ac8a0b30 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:27.767743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581cd94-8c6a-435b-b478-52b6ac8a0b30 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:27.768189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581cd94-8c6a-435b-b478-52b6ac8a0b30 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:27.769998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 157/296] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:27.780417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6c551-6ad3-47a9-87a5-8172d9eb873c None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:27.781485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6c551-6ad3-47a9-87a5-8172d9eb873c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:27.783075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6c551-6ad3-47a9-87a5-8172d9eb873c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:27.793389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 140/297] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:27.815384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5ac6c551-6ad3-47a9-87a5-8172d9eb873c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:27.817849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5ac6c551-6ad3-47a9-87a5-8172d9eb873c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:27.906626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 158/298] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:28.016607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2c8bc-7158-4f1e-b591-d157b491bdee None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:28.017393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2c8bc-7158-4f1e-b591-d157b491bdee None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:28.017875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2c8bc-7158-4f1e-b591-d157b491bdee None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:28.019769 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 141/299] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:28.025940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc77492-535e-4e9c-9d1c-3f6f55f57613 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:28.026517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc77492-535e-4e9c-9d1c-3f6f55f57613 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:28.026994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc77492-535e-4e9c-9d1c-3f6f55f57613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:28.046959 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-acc77492-535e-4e9c-9d1c-3f6f55f57613 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:28.049007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-acc77492-535e-4e9c-9d1c-3f6f55f57613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:28.078485 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2c8bc-7158-4f1e-b591-d157b491bdee None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:28.136423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 159/300] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:28.189745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:28.194982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc77492-535e-4e9c-9d1c-3f6f55f57613 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:28.210386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:28.215569 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2c8bc-7158-4f1e-b591-d157b491bdee None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:28.256128 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461520c6-6684-454c-a8c5-bf9b65cf96cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:28.257040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461520c6-6684-454c-a8c5-bf9b65cf96cd None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:28.257489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461520c6-6684-454c-a8c5-bf9b65cf96cd None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:28.257918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461520c6-6684-454c-a8c5-bf9b65cf96cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:28.269647 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461520c6-6684-454c-a8c5-bf9b65cf96cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-461520c6-6684-454c-a8c5-bf9b65cf96cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:28.274944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:28.276226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:28.277010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:28.277010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:28.277010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:28.277010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:28.277010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:28.277010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:28.277010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:28.277010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:28.277010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:28.277010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:28.277010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:28.277010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:28.278351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 142/301] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:28.286357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461520c6-6684-454c-a8c5-bf9b65cf96cd None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:28.328755 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc77492-535e-4e9c-9d1c-3f6f55f57613 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:28.378976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5d5d9-9fc1-4455-a389-89c1ec0977d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:28.380025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5d5d9-9fc1-4455-a389-89c1ec0977d8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:28.380516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5d5d9-9fc1-4455-a389-89c1ec0977d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:28.381006 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5d5d9-9fc1-4455-a389-89c1ec0977d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:28.396837 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5d5d9-9fc1-4455-a389-89c1ec0977d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-bdf5d5d9-9fc1-4455-a389-89c1ec0977d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:28.404060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:28.404950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:28.405764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:28.405764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:28.405764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:28.405764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:28.405764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:28.405764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:28.405764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:28.405764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:28.405764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:28.405764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:28.405764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:28.405764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:28.409163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 160/302] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:28.412796 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5d5d9-9fc1-4455-a389-89c1ec0977d8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:28.421408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461520c6-6684-454c-a8c5-bf9b65cf96cd None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:28.467316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a1b2e-b773-4d68-b307-47c90ad48993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:28.468273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a1b2e-b773-4d68-b307-47c90ad48993 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:28.468750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a1b2e-b773-4d68-b307-47c90ad48993 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:28.469257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a1b2e-b773-4d68-b307-47c90ad48993 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:28.470570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a1b2e-b773-4d68-b307-47c90ad48993 None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:28.470879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a1b2e-b773-4d68-b307-47c90ad48993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:28.472259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a1b2e-b773-4d68-b307-47c90ad48993 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:28.473908 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:28.487140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 143/303] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:28.488095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a1b2e-b773-4d68-b307-47c90ad48993 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:28.500189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:28.546601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5d5d9-9fc1-4455-a389-89c1ec0977d8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:28.587114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f3ab1-6f3b-47fc-9ebb-81b6ed042c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:28.588062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f3ab1-6f3b-47fc-9ebb-81b6ed042c23 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:28.588519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f3ab1-6f3b-47fc-9ebb-81b6ed042c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:28.588982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f3ab1-6f3b-47fc-9ebb-81b6ed042c23 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:28.589849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f3ab1-6f3b-47fc-9ebb-81b6ed042c23 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:28.590326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f3ab1-6f3b-47fc-9ebb-81b6ed042c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:28.591995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f3ab1-6f3b-47fc-9ebb-81b6ed042c23 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:28.601924 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 161/304] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:28.603216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f3ab1-6f3b-47fc-9ebb-81b6ed042c23 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:28.616301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a1b2e-b773-4d68-b307-47c90ad48993 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:28.662812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b8df5-ea89-47a1-900c-d965be808d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:28.663617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b8df5-ea89-47a1-900c-d965be808d1d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:28.663904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b8df5-ea89-47a1-900c-d965be808d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:28.664474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b8df5-ea89-47a1-900c-d965be808d1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:28.665684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b8df5-ea89-47a1-900c-d965be808d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:28.667616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b8df5-ea89-47a1-900c-d965be808d1d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:28.669837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 144/305] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:28.670712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b8df5-ea89-47a1-900c-d965be808d1d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:28.723334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f3ab1-6f3b-47fc-9ebb-81b6ed042c23 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:28.739204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:28.756509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:28.764339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70851e-057d-4316-97cb-2c81805ad083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:28.765299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70851e-057d-4316-97cb-2c81805ad083 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:28.765580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70851e-057d-4316-97cb-2c81805ad083 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:28.765859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70851e-057d-4316-97cb-2c81805ad083 None admin] PATH_INFO: `/v3/users/566a8ff23a9940c392826071f5ff7295` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:28.767029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70851e-057d-4316-97cb-2c81805ad083 None admin] RBAC: Authorizing `identity:get_user(user_id=566a8ff23a9940c392826071f5ff7295)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:28.768960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70851e-057d-4316-97cb-2c81805ad083 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:28.776232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 162/306] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:28 2022] GET /identity/v3/users/566a8ff23a9940c392826071f5ff7295 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:28.776232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70851e-057d-4316-97cb-2c81805ad083 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:28.785697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b8df5-ea89-47a1-900c-d965be808d1d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:28.822987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c244c6-1d4f-43eb-9ac1-f87a3a4d604e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:28.823606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c244c6-1d4f-43eb-9ac1-f87a3a4d604e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:28.823883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c244c6-1d4f-43eb-9ac1-f87a3a4d604e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:28.824153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c244c6-1d4f-43eb-9ac1-f87a3a4d604e None admin] PATH_INFO: `/v3/users/9387a2657405430591671b2eb24aaf3d` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:28.838263 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c244c6-1d4f-43eb-9ac1-f87a3a4d604e None admin] RBAC: Authorizing `identity:get_user(user_id=9387a2657405430591671b2eb24aaf3d)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:28.839651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c244c6-1d4f-43eb-9ac1-f87a3a4d604e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:28.841322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 145/307] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:28 2022] GET /identity/v3/users/9387a2657405430591671b2eb24aaf3d => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:28.842075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c244c6-1d4f-43eb-9ac1-f87a3a4d604e None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:28.893354 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70851e-057d-4316-97cb-2c81805ad083 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:28.930561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8c6f4-cc54-4f45-964a-cb9316784f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:28.931179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8c6f4-cc54-4f45-964a-cb9316784f3c None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:28.931460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8c6f4-cc54-4f45-964a-cb9316784f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:28.931742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8c6f4-cc54-4f45-964a-cb9316784f3c None admin] PATH_INFO: `/v3/groups` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:28.932376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8c6f4-cc54-4f45-964a-cb9316784f3c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:28.933845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8c6f4-cc54-4f45-964a-cb9316784f3c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:28.950832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 163/308] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Sep 28 08:24:28 2022] POST /identity/v3/groups => generated 229 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:24:28.951176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c244c6-1d4f-43eb-9ac1-f87a3a4d604e None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:28.951654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8c6f4-cc54-4f45-964a-cb9316784f3c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:28.997474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:29.005807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbf59c-b60c-4278-b211-83e0c4b743b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:29.006841 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbf59c-b60c-4278-b211-83e0c4b743b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:29.007370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbf59c-b60c-4278-b211-83e0c4b743b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:29.008026 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbf59c-b60c-4278-b211-83e0c4b743b4 None admin] PATH_INFO: `/v3/system/users/566a8ff23a9940c392826071f5ff7295/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:29.010218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbf59c-b60c-4278-b211-83e0c4b743b4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=566a8ff23a9940c392826071f5ff7295, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:29.011949 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbf59c-b60c-4278-b211-83e0c4b743b4 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:29.016881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:29.033132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 146/309] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Sep 28 08:24:28 2022] PUT /identity/v3/system/users/566a8ff23a9940c392826071f5ff7295/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:29.074644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8c6f4-cc54-4f45-964a-cb9316784f3c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:29.118116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37455c27-af5a-42b4-8e56-c1b5828bb308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:29.118638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37455c27-af5a-42b4-8e56-c1b5828bb308 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:29.118920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37455c27-af5a-42b4-8e56-c1b5828bb308 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:29.119192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37455c27-af5a-42b4-8e56-c1b5828bb308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:29.119964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37455c27-af5a-42b4-8e56-c1b5828bb308 None admin] RBAC: Adding query filter params (role.id=718dd530c67f4a1891e292e729330f51, user.id=9387a2657405430591671b2eb24aaf3d, scope.system=all) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:29.120275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37455c27-af5a-42b4-8e56-c1b5828bb308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:29.121814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37455c27-af5a-42b4-8e56-c1b5828bb308 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:29.128321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 164/310] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Sep 28 08:24:28 2022] GET /identity/v3/role_assignments?user.id=9387a2657405430591671b2eb24aaf3d&scope.system=all&role.id=718dd530c67f4a1891e292e729330f51 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:29.283179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beb217-3524-4b23-b7a4-40d5a02fc14e None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:29.283476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beb217-3524-4b23-b7a4-40d5a02fc14e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:29.283757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7beb217-3524-4b23-b7a4-40d5a02fc14e None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:29.285272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 147/311] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:29.291331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6b538-862e-459e-b66f-6bffa1967092 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:29.291949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6b538-862e-459e-b66f-6bffa1967092 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:29.292452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6b538-862e-459e-b66f-6bffa1967092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:29.314512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-fdb6b538-862e-459e-b66f-6bffa1967092 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:29.316076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-fdb6b538-862e-459e-b66f-6bffa1967092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:29.393610 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 165/312] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:29.678822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7beb217-3524-4b23-b7a4-40d5a02fc14e None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:29.785784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7beb217-3524-4b23-b7a4-40d5a02fc14e None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:29.827525 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a71318-fc40-431b-84e6-75ca001db8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:29.828183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a71318-fc40-431b-84e6-75ca001db8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:29.828464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a71318-fc40-431b-84e6-75ca001db8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:29.828743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a71318-fc40-431b-84e6-75ca001db8e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:29.841920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a71318-fc40-431b-84e6-75ca001db8e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-e6a71318-fc40-431b-84e6-75ca001db8e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:29.846999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:29.847952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:29.848734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 148/313] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:29.855993 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6b538-862e-459e-b66f-6bffa1967092 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:29.914982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:29.996781 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6b538-862e-459e-b66f-6bffa1967092 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:30.045723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f98ab-7440-44c5-ae30-1a3b4b713d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:30.046808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f98ab-7440-44c5-ae30-1a3b4b713d13 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:30.047300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f98ab-7440-44c5-ae30-1a3b4b713d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:30.047793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f98ab-7440-44c5-ae30-1a3b4b713d13 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:30.049625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f98ab-7440-44c5-ae30-1a3b4b713d13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:30.050010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f98ab-7440-44c5-ae30-1a3b4b713d13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:30.052724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f98ab-7440-44c5-ae30-1a3b4b713d13 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:30.067876 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 166/314] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:30.073792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a71318-fc40-431b-84e6-75ca001db8e3 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:30.147816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:30.255659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a71318-fc40-431b-84e6-75ca001db8e3 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:30.295686 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1538a30-b3dc-4bfe-9949-8c5888246255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:30.296373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1538a30-b3dc-4bfe-9949-8c5888246255 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:30.296654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1538a30-b3dc-4bfe-9949-8c5888246255 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:30.296924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1538a30-b3dc-4bfe-9949-8c5888246255 None admin] PATH_INFO: `/v3/users/3aa75836e72d443980eef361a42897a8` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:30.298144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1538a30-b3dc-4bfe-9949-8c5888246255 None admin] RBAC: Authorizing `identity:get_user(user_id=3aa75836e72d443980eef361a42897a8)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:30.299636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1538a30-b3dc-4bfe-9949-8c5888246255 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:30.301657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 149/315] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:30 2022] GET /identity/v3/users/3aa75836e72d443980eef361a42897a8 => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:30.310369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f98ab-7440-44c5-ae30-1a3b4b713d13 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:30.428882 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f98ab-7440-44c5-ae30-1a3b4b713d13 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:30.444518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:30.483696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ab630-86da-4b8a-b953-84dd833de83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:30.484320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ab630-86da-4b8a-b953-84dd833de83c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:30.484595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ab630-86da-4b8a-b953-84dd833de83c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:30.484874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ab630-86da-4b8a-b953-84dd833de83c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:30.567975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ab630-86da-4b8a-b953-84dd833de83c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:30.569543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ab630-86da-4b8a-b953-84dd833de83c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:30.674808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 167/316] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:30.680849 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1538a30-b3dc-4bfe-9949-8c5888246255 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:30.821659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1538a30-b3dc-4bfe-9949-8c5888246255 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:30.849700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:30.884222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da6b29-2d77-4617-a77d-4b39e512b18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:30.885333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da6b29-2d77-4617-a77d-4b39e512b18e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:30.885824 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da6b29-2d77-4617-a77d-4b39e512b18e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:30.886311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da6b29-2d77-4617-a77d-4b39e512b18e None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:30.888218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da6b29-2d77-4617-a77d-4b39e512b18e None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:30.890852 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da6b29-2d77-4617-a77d-4b39e512b18e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:30.894859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 150/317] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:30 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:30.900992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ab630-86da-4b8a-b953-84dd833de83c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:30.921468 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:31.017946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ab630-86da-4b8a-b953-84dd833de83c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:31.057978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd00c5-28f3-476b-b413-4223a78eb579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:31.058607 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd00c5-28f3-476b-b413-4223a78eb579 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:31.058972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd00c5-28f3-476b-b413-4223a78eb579 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:31.059216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd00c5-28f3-476b-b413-4223a78eb579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:31.059953 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd00c5-28f3-476b-b413-4223a78eb579 None admin] RBAC: Adding query filter params (role.id=718dd530c67f4a1891e292e729330f51, scope.project.id=53c2a41061214641a61babeb674afeae, user.id=3aa75836e72d443980eef361a42897a8) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:31.060300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd00c5-28f3-476b-b413-4223a78eb579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:31.061834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd00c5-28f3-476b-b413-4223a78eb579 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:31.069470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 168/318] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:24:30 2022] GET /identity/v3/role_assignments?user.id=3aa75836e72d443980eef361a42897a8&scope.project.id=53c2a41061214641a61babeb674afeae&role.id=718dd530c67f4a1891e292e729330f51 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:31.469304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a99fb-4e2a-4555-941c-fda951049344 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:31.469675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a99fb-4e2a-4555-941c-fda951049344 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:31.469868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a99fb-4e2a-4555-941c-fda951049344 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:31.472251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 151/319] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:31.473810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa6ce0-f5a5-4ec4-ad6a-80d95c00a57f None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:31.474098 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa6ce0-f5a5-4ec4-ad6a-80d95c00a57f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:31.474445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa6ce0-f5a5-4ec4-ad6a-80d95c00a57f None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:31.476872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 169/320] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:31.477237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1722bc-355c-4e26-a7ce-777cc5b1fe78 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:31.477749 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1722bc-355c-4e26-a7ce-777cc5b1fe78 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:31.478172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1722bc-355c-4e26-a7ce-777cc5b1fe78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:31.483116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14440a7e-c5f3-49c5-95b6-d29c4c543550 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:31.483395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14440a7e-c5f3-49c5-95b6-d29c4c543550 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:31.483672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14440a7e-c5f3-49c5-95b6-d29c4c543550 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:31.484976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 170/321] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:31.487933 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b23e01-9ae9-4e96-b241-b9b118bc39d2 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:31.488244 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b23e01-9ae9-4e96-b241-b9b118bc39d2 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:31.488525 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b23e01-9ae9-4e96-b241-b9b118bc39d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:31.499853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ca1722bc-355c-4e26-a7ce-777cc5b1fe78 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:31.501123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ca1722bc-355c-4e26-a7ce-777cc5b1fe78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:31.509543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-85b23e01-9ae9-4e96-b241-b9b118bc39d2 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:31.510725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-85b23e01-9ae9-4e96-b241-b9b118bc39d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:31.568944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 152/322] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:31.570390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a1f7f-49cc-4793-93d9-e75eda86f8fa None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:31.570683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a1f7f-49cc-4793-93d9-e75eda86f8fa None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:31.570961 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a1f7f-49cc-4793-93d9-e75eda86f8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:31.583511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 171/323] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:31.590075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-3c1a1f7f-49cc-4793-93d9-e75eda86f8fa None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:31.591282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3c1a1f7f-49cc-4793-93d9-e75eda86f8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:31.666358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 153/324] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:31.858130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b23e01-9ae9-4e96-b241-b9b118bc39d2 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:31.858461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a1f7f-49cc-4793-93d9-e75eda86f8fa None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:31.868496 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:31.907246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:31.964858 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b23e01-9ae9-4e96-b241-b9b118bc39d2 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:31.969447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a1f7f-49cc-4793-93d9-e75eda86f8fa None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:32.009501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87f5e9-ba6a-4f18-91e7-c60fb0d3f37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:32.010134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87f5e9-ba6a-4f18-91e7-c60fb0d3f37d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:32.010409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87f5e9-ba6a-4f18-91e7-c60fb0d3f37d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:32.010676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87f5e9-ba6a-4f18-91e7-c60fb0d3f37d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:32.013243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1037c-913a-442c-85d7-4863d1049e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:32.013900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1037c-913a-442c-85d7-4863d1049e53 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:32.014211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1037c-913a-442c-85d7-4863d1049e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:32.014482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1037c-913a-442c-85d7-4863d1049e53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:32.022455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87f5e9-ba6a-4f18-91e7-c60fb0d3f37d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:32.027199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1037c-913a-442c-85d7-4863d1049e53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-9d87f5e9-ba6a-4f18-91e7-c60fb0d3f37d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:32.027624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:32.029922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 172/325] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:32.030823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87f5e9-ba6a-4f18-91e7-c60fb0d3f37d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-50a1037c-913a-442c-85d7-4863d1049e53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:32.032231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:32.033158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 154/326] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:32.038099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1037c-913a-442c-85d7-4863d1049e53 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:32.109956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:32.147074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1037c-913a-442c-85d7-4863d1049e53 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:32.147547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87f5e9-ba6a-4f18-91e7-c60fb0d3f37d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:32.148693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:32.186274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdf7be-3a6f-43c9-8f31-c4091f156ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:32.187114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdf7be-3a6f-43c9-8f31-c4091f156ecc None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:32.187455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdf7be-3a6f-43c9-8f31-c4091f156ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:32.187727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdf7be-3a6f-43c9-8f31-c4091f156ecc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:32.189415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdf7be-3a6f-43c9-8f31-c4091f156ecc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:32.190878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdf7be-3a6f-43c9-8f31-c4091f156ecc None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:32.191051 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afd5aa-785c-4a7d-864d-4aaef8099a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:32.191672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afd5aa-785c-4a7d-864d-4aaef8099a61 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:32.191947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afd5aa-785c-4a7d-864d-4aaef8099a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:32.192213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afd5aa-785c-4a7d-864d-4aaef8099a61 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:32.192478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 173/327] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:32.193249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afd5aa-785c-4a7d-864d-4aaef8099a61 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:32.193249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afd5aa-785c-4a7d-864d-4aaef8099a61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:32.193384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdf7be-3a6f-43c9-8f31-c4091f156ecc None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:32.194686 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afd5aa-785c-4a7d-864d-4aaef8099a61 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:32.204140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 155/328] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:32.205615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afd5aa-785c-4a7d-864d-4aaef8099a61 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:32.316198 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdf7be-3a6f-43c9-8f31-c4091f156ecc None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:32.323520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afd5aa-785c-4a7d-864d-4aaef8099a61 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:32.353758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab420ef-6589-4c74-a6d7-a3e75a6eaca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:32.354376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab420ef-6589-4c74-a6d7-a3e75a6eaca5 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:32.354658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab420ef-6589-4c74-a6d7-a3e75a6eaca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:32.354951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab420ef-6589-4c74-a6d7-a3e75a6eaca5 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:32.355617 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab420ef-6589-4c74-a6d7-a3e75a6eaca5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:32.355921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab420ef-6589-4c74-a6d7-a3e75a6eaca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:32.357330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab420ef-6589-4c74-a6d7-a3e75a6eaca5 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:32.359361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898279e-3893-4790-bc2b-8a95dd2f29f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:32.359691 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:32.360025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898279e-3893-4790-bc2b-8a95dd2f29f9 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:32.360244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898279e-3893-4790-bc2b-8a95dd2f29f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:32.361173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898279e-3893-4790-bc2b-8a95dd2f29f9 None admin] PATH_INFO: `/v3/groups` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:32.361757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898279e-3893-4790-bc2b-8a95dd2f29f9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:32.363171 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898279e-3893-4790-bc2b-8a95dd2f29f9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:32.367113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 174/329] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:32.367918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab420ef-6589-4c74-a6d7-a3e75a6eaca5 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:32.384891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 156/330] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Sep 28 08:24:32 2022] POST /identity/v3/groups => generated 226 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:24:32.385742 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898279e-3893-4790-bc2b-8a95dd2f29f9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:32.413553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:32.498582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898279e-3893-4790-bc2b-8a95dd2f29f9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:32.500273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab420ef-6589-4c74-a6d7-a3e75a6eaca5 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:32.545775 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732da01-724a-4da0-9321-7f7f039073f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:32.546429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732da01-724a-4da0-9321-7f7f039073f7 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:32.546707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732da01-724a-4da0-9321-7f7f039073f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:32.546984 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732da01-724a-4da0-9321-7f7f039073f7 None admin] PATH_INFO: `/v3/users/566a8ff23a9940c392826071f5ff7295` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:32.548723 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732da01-724a-4da0-9321-7f7f039073f7 None admin] RBAC: Authorizing `identity:get_user(user_id=566a8ff23a9940c392826071f5ff7295)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:32.550241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732da01-724a-4da0-9321-7f7f039073f7 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:32.551865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 157/331] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:32 2022] GET /identity/v3/users/566a8ff23a9940c392826071f5ff7295 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:32.553938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f867a-95df-44c3-b81e-8ccedf7263d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:32.554575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f867a-95df-44c3-b81e-8ccedf7263d0 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:32.554853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f867a-95df-44c3-b81e-8ccedf7263d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:32.555126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f867a-95df-44c3-b81e-8ccedf7263d0 None admin] PATH_INFO: `/v3/users/9387a2657405430591671b2eb24aaf3d` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:32.556266 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f867a-95df-44c3-b81e-8ccedf7263d0 None admin] RBAC: Authorizing `identity:get_user(user_id=9387a2657405430591671b2eb24aaf3d)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:32.557737 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f867a-95df-44c3-b81e-8ccedf7263d0 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:32.557858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732da01-724a-4da0-9321-7f7f039073f7 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:32.559433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 175/332] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:32 2022] GET /identity/v3/users/9387a2657405430591671b2eb24aaf3d => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:32.565282 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f867a-95df-44c3-b81e-8ccedf7263d0 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:32.637217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:32.677780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:32.685250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f867a-95df-44c3-b81e-8ccedf7263d0 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:32.693900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732da01-724a-4da0-9321-7f7f039073f7 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:32.724728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c73f2f-e23f-4a91-80c1-9701318f6341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:32.725424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c73f2f-e23f-4a91-80c1-9701318f6341 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:32.725705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c73f2f-e23f-4a91-80c1-9701318f6341 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:32.725979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c73f2f-e23f-4a91-80c1-9701318f6341 None admin] PATH_INFO: `/v3/system/users/9387a2657405430591671b2eb24aaf3d/roles/718dd530c67f4a1891e292e729330f51` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:32.727400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c73f2f-e23f-4a91-80c1-9701318f6341 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9387a2657405430591671b2eb24aaf3d, role_id=718dd530c67f4a1891e292e729330f51)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:32.728757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c73f2f-e23f-4a91-80c1-9701318f6341 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:32.733986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a9a72-0a85-4afd-958f-f8a114234c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:32.734693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a9a72-0a85-4afd-958f-f8a114234c0f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:32.735041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a9a72-0a85-4afd-958f-f8a114234c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:32.735375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a9a72-0a85-4afd-958f-f8a114234c0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:32.736856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a9a72-0a85-4afd-958f-f8a114234c0f None admin] RBAC: Adding query filter params (role.id=4c80373ee4c34171a7f29e5ab6c09ce0, user.id=566a8ff23a9940c392826071f5ff7295, scope.system=all) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:32.737733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a9a72-0a85-4afd-958f-f8a114234c0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:32.739320 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a9a72-0a85-4afd-958f-f8a114234c0f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:32.740941 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 176/333] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Sep 28 08:24:32 2022] PUT /identity/v3/system/users/9387a2657405430591671b2eb24aaf3d/roles/718dd530c67f4a1891e292e729330f51 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:32.754856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 158/334] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Sep 28 08:24:32 2022] GET /identity/v3/role_assignments?user.id=566a8ff23a9940c392826071f5ff7295&scope.system=all&role.id=4c80373ee4c34171a7f29e5ab6c09ce0 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:35.103089 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de290-1483-46e0-9639-dfafa1bffb55 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:35.104171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de290-1483-46e0-9639-dfafa1bffb55 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:35.104661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de290-1483-46e0-9639-dfafa1bffb55 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:35.106660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 177/335] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:35.115421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48207535-886e-4e24-9992-20eddcff8dc7 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:35.116582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48207535-886e-4e24-9992-20eddcff8dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:35.117080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48207535-886e-4e24-9992-20eddcff8dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:35.142737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-48207535-886e-4e24-9992-20eddcff8dc7 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:35.144301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-48207535-886e-4e24-9992-20eddcff8dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:35.222014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 159/336] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:35.407240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bfdf5-2e74-457e-8939-9978d6a739fa None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:35.408550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bfdf5-2e74-457e-8939-9978d6a739fa None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:35.409054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bfdf5-2e74-457e-8939-9978d6a739fa None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:35.410930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 178/337] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:35.416256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2c99f-dcca-4370-ac84-90b1bb3e8a4e None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:35.416710 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2c99f-dcca-4370-ac84-90b1bb3e8a4e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:35.416999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2c99f-dcca-4370-ac84-90b1bb3e8a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:35.441709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-4bc2c99f-dcca-4370-ac84-90b1bb3e8a4e None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:35.442905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4bc2c99f-dcca-4370-ac84-90b1bb3e8a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:35.540972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 160/338] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:35.717863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bfdf5-2e74-457e-8939-9978d6a739fa None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:35.853024 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bfdf5-2e74-457e-8939-9978d6a739fa None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:35.888153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:35.897865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591c324-cfbe-4e45-90e9-65772f5a0f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:35.898547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591c324-cfbe-4e45-90e9-65772f5a0f2d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:35.898836 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591c324-cfbe-4e45-90e9-65772f5a0f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:35.900987 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591c324-cfbe-4e45-90e9-65772f5a0f2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:35.914738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591c324-cfbe-4e45-90e9-65772f5a0f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-a591c324-cfbe-4e45-90e9-65772f5a0f2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:35.919819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:35.920763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:35.921625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 179/339] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:35.927946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2c99f-dcca-4370-ac84-90b1bb3e8a4e None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:35.941913 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:35.950962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb9bf3-c38d-4f8b-8918-7be3cb3f4029 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:35.951272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb9bf3-c38d-4f8b-8918-7be3cb3f4029 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:35.951497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb9bf3-c38d-4f8b-8918-7be3cb3f4029 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:35.953025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 180/340] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:24:35.958834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd9f8-3745-4935-ab97-a564fb9c51ee None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:35.959100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd9f8-3745-4935-ab97-a564fb9c51ee None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:35.959370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd9f8-3745-4935-ab97-a564fb9c51ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:35.984182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-685cd9f8-3745-4935-ab97-a564fb9c51ee None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:35.987428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-685cd9f8-3745-4935-ab97-a564fb9c51ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:36.123562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2c99f-dcca-4370-ac84-90b1bb3e8a4e None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:36.137113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 181/341] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:36.137995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cd9f8-3745-4935-ab97-a564fb9c51ee None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:36.166031 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80029e9b-8871-43fd-91ad-3056fa46a51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:36.166908 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80029e9b-8871-43fd-91ad-3056fa46a51e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:36.167252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80029e9b-8871-43fd-91ad-3056fa46a51e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:36.167532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80029e9b-8871-43fd-91ad-3056fa46a51e None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:36.168209 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80029e9b-8871-43fd-91ad-3056fa46a51e None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:36.168512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80029e9b-8871-43fd-91ad-3056fa46a51e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:36.170452 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80029e9b-8871-43fd-91ad-3056fa46a51e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:36.184315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 161/342] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:36.191618 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80029e9b-8871-43fd-91ad-3056fa46a51e None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:36.280916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:36.318387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80029e9b-8871-43fd-91ad-3056fa46a51e None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:36.359438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f75741-c11f-44f8-8272-8f47f7b14a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:36.360109 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f75741-c11f-44f8-8272-8f47f7b14a04 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:36.360668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f75741-c11f-44f8-8272-8f47f7b14a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:36.360940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f75741-c11f-44f8-8272-8f47f7b14a04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:36.423807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cd9f8-3745-4935-ab97-a564fb9c51ee None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:36.444821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f75741-c11f-44f8-8272-8f47f7b14a04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:36.447084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f75741-c11f-44f8-8272-8f47f7b14a04 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:36.484985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:36.539370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:36.555527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04de60-9933-4527-9816-2573e50b6234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:36.556176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04de60-9933-4527-9816-2573e50b6234 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:36.556454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04de60-9933-4527-9816-2573e50b6234 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:36.556723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04de60-9933-4527-9816-2573e50b6234 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:36.576165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 162/343] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:36.580942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120329d-b539-45a8-bfea-7fb535dce6ba None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:36.581268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120329d-b539-45a8-bfea-7fb535dce6ba None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:36.581550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120329d-b539-45a8-bfea-7fb535dce6ba None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:36.583118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 163/344] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:36.586132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120329d-b539-45a8-bfea-7fb535dce6ba None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:36.593761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04de60-9933-4527-9816-2573e50b6234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-bd04de60-9933-4527-9816-2573e50b6234 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:36.599562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:36.600452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:36.601300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:36.601300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:36.601300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:36.601300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:36.601300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:36.601300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:36.601300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:36.601300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:36.601300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:36.601300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:36.601300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:36.601300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:36.601909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 182/345] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:24:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 463 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:36.602978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017eb06-d137-49c2-a20b-6bddc2a1e3e0 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:36.603263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017eb06-d137-49c2-a20b-6bddc2a1e3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:36.603538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017eb06-d137-49c2-a20b-6bddc2a1e3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:36.634952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-c017eb06-d137-49c2-a20b-6bddc2a1e3e0 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:36.636176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c017eb06-d137-49c2-a20b-6bddc2a1e3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:36.722094 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120329d-b539-45a8-bfea-7fb535dce6ba None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:36.722290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 183/346] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:36.723109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017eb06-d137-49c2-a20b-6bddc2a1e3e0 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:36.762228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6d6be-47d7-4104-b18a-da3c0ad821ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:36.763245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6d6be-47d7-4104-b18a-da3c0ad821ff None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:36.763799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6d6be-47d7-4104-b18a-da3c0ad821ff None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:36.764285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6d6be-47d7-4104-b18a-da3c0ad821ff None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:36.770019 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c6d6be-47d7-4104-b18a-da3c0ad821ff None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:36.771861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c6d6be-47d7-4104-b18a-da3c0ad821ff None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:36.774345 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 164/347] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:36 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:36.775342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6d6be-47d7-4104-b18a-da3c0ad821ff None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:36.853828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:36.871443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017eb06-d137-49c2-a20b-6bddc2a1e3e0 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:36.885783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:36.915065 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e2574-a10b-4f24-8a4b-d503e199eec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:36.915698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e2574-a10b-4f24-8a4b-d503e199eec9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:36.915975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e2574-a10b-4f24-8a4b-d503e199eec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:36.916248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e2574-a10b-4f24-8a4b-d503e199eec9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:36.928466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e2574-a10b-4f24-8a4b-d503e199eec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-b10e2574-a10b-4f24-8a4b-d503e199eec9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:36.933396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:36.934309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:36.935175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 184/348] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:36.936299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e2574-a10b-4f24-8a4b-d503e199eec9 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:36.959827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6d6be-47d7-4104-b18a-da3c0ad821ff None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:37.012251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16546ba-443d-45d3-8471-0b1537bf2cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:37.013342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16546ba-443d-45d3-8471-0b1537bf2cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:37.014117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16546ba-443d-45d3-8471-0b1537bf2cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:37.014643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16546ba-443d-45d3-8471-0b1537bf2cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:37.015657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16546ba-443d-45d3-8471-0b1537bf2cb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:37.016181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16546ba-443d-45d3-8471-0b1537bf2cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:37.018104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16546ba-443d-45d3-8471-0b1537bf2cb1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:37.028829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 165/349] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:37.030704 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931b476-1eb4-4fe7-bb90-1b169b103f36 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:37.031212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931b476-1eb4-4fe7-bb90-1b169b103f36 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:37.031688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931b476-1eb4-4fe7-bb90-1b169b103f36 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:37.037678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 166/350] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:37.038932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931b476-1eb4-4fe7-bb90-1b169b103f36 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:37.070531 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e2574-a10b-4f24-8a4b-d503e199eec9 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:37.108506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6b62d-3e91-4316-ac58-92866a557031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:37.109159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6b62d-3e91-4316-ac58-92866a557031 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:37.109435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6b62d-3e91-4316-ac58-92866a557031 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:37.109711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6b62d-3e91-4316-ac58-92866a557031 None admin] PATH_INFO: `/v3/groups/03e5023068a54913a7187468f0fb19bc` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:37.117000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6b62d-3e91-4316-ac58-92866a557031 None admin] RBAC: Authorizing `identity:get_group(group_id=03e5023068a54913a7187468f0fb19bc)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:37.118520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6b62d-3e91-4316-ac58-92866a557031 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:37.120571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 185/351] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:36 2022] GET /identity/v3/groups/03e5023068a54913a7187468f0fb19bc => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:37.121926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1ea68-db4e-4dcf-bd43-c52199cc33a6 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:37.122209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1ea68-db4e-4dcf-bd43-c52199cc33a6 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:37.122490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1ea68-db4e-4dcf-bd43-c52199cc33a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:37.144134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b4e1ea68-db4e-4dcf-bd43-c52199cc33a6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:37.145299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b4e1ea68-db4e-4dcf-bd43-c52199cc33a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:37.171333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931b476-1eb4-4fe7-bb90-1b169b103f36 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:37.174424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:37.182937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:37.213941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f765c-2ca6-4710-828d-8fcaaaf50027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:37.214563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f765c-2ca6-4710-828d-8fcaaaf50027 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:37.214843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f765c-2ca6-4710-828d-8fcaaaf50027 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:37.215112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f765c-2ca6-4710-828d-8fcaaaf50027 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:37.216249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f765c-2ca6-4710-828d-8fcaaaf50027 None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:37.216678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f765c-2ca6-4710-828d-8fcaaaf50027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:37.217966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f765c-2ca6-4710-828d-8fcaaaf50027 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:37.227045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 167/352] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:37.227897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f765c-2ca6-4710-828d-8fcaaaf50027 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:37.237415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 186/353] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:37.238264 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1ea68-db4e-4dcf-bd43-c52199cc33a6 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:37.342392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f765c-2ca6-4710-828d-8fcaaaf50027 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:37.373963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1ea68-db4e-4dcf-bd43-c52199cc33a6 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:37.386879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ab6ec-0edb-47d4-833d-cdfbb6198903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:37.387507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ab6ec-0edb-47d4-833d-cdfbb6198903 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:37.388005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ab6ec-0edb-47d4-833d-cdfbb6198903 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:37.388118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ab6ec-0edb-47d4-833d-cdfbb6198903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:37.426524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55deb8fb-92f6-420f-832c-2b2ac40fcc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:37.427143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deb8fb-92f6-420f-832c-2b2ac40fcc63 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:37.427419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deb8fb-92f6-420f-832c-2b2ac40fcc63 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:37.427686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deb8fb-92f6-420f-832c-2b2ac40fcc63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:37.428699 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55deb8fb-92f6-420f-832c-2b2ac40fcc63 None admin] RBAC: Adding query filter params (group.id=03e5023068a54913a7187468f0fb19bc, role.id=4c80373ee4c34171a7f29e5ab6c09ce0, scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:37.428699 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55deb8fb-92f6-420f-832c-2b2ac40fcc63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:37.430218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55deb8fb-92f6-420f-832c-2b2ac40fcc63 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:37.431838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:37.436129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 187/354] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 08:24:37 2022] GET /identity/v3/role_assignments?group.id=03e5023068a54913a7187468f0fb19bc&scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859&role.id=4c80373ee4c34171a7f29e5ab6c09ce0 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:37.438169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55deb8fb-92f6-420f-832c-2b2ac40fcc63 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:37.474631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:37.483609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ab6ec-0edb-47d4-833d-cdfbb6198903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:37.486198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ab6ec-0edb-47d4-833d-cdfbb6198903 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:37.577344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55deb8fb-92f6-420f-832c-2b2ac40fcc63 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:37.622628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 168/355] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:37.624352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cff64-9192-411d-a018-e9168ce02b24 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:37.624576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cff64-9192-411d-a018-e9168ce02b24 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:37.624940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cff64-9192-411d-a018-e9168ce02b24 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:37.626581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 169/356] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:37.628014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cff64-9192-411d-a018-e9168ce02b24 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:37.641602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fe878-0b81-4ffe-889f-480dd579a259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:37.643547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fe878-0b81-4ffe-889f-480dd579a259 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:37.644003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fe878-0b81-4ffe-889f-480dd579a259 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:37.644282 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fe878-0b81-4ffe-889f-480dd579a259 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:37.661739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fe878-0b81-4ffe-889f-480dd579a259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-b84fe878-0b81-4ffe-889f-480dd579a259 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:37.667153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:37.668042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:37.668814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 188/357] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:37.669568 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fe878-0b81-4ffe-889f-480dd579a259 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:37.718524 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:37.776047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cff64-9192-411d-a018-e9168ce02b24 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:37.789303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:37.807127 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fe878-0b81-4ffe-889f-480dd579a259 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:37.828019 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b3fbf-5a22-41ee-a410-abe942c7ae21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:37.828965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b3fbf-5a22-41ee-a410-abe942c7ae21 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:37.829360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b3fbf-5a22-41ee-a410-abe942c7ae21 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:37.829635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b3fbf-5a22-41ee-a410-abe942c7ae21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:37.849816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c0160-e3b1-4467-8131-2bbbfe7778b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:37.850389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c0160-e3b1-4467-8131-2bbbfe7778b9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:37.850689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c0160-e3b1-4467-8131-2bbbfe7778b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:37.850970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c0160-e3b1-4467-8131-2bbbfe7778b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:37.865228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c0160-e3b1-4467-8131-2bbbfe7778b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-8a4c0160-e3b1-4467-8131-2bbbfe7778b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:37.870070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:37.870988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 189/358] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:24:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:37.872163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c0160-e3b1-4467-8131-2bbbfe7778b9 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:37.936543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b3fbf-5a22-41ee-a410-abe942c7ae21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:37.939680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b3fbf-5a22-41ee-a410-abe942c7ae21 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:38.021203 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:38.038078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c0160-e3b1-4467-8131-2bbbfe7778b9 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:38.077367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 170/359] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:38.078808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5b913-0951-4d47-ae22-c83a85904017 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:38.079258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5b913-0951-4d47-ae22-c83a85904017 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:38.079631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5b913-0951-4d47-ae22-c83a85904017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:38.085913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b70e54-f161-4a24-89d8-6a57c854de92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:38.086925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b70e54-f161-4a24-89d8-6a57c854de92 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:38.088171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b70e54-f161-4a24-89d8-6a57c854de92 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:38.088604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b70e54-f161-4a24-89d8-6a57c854de92 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:38.089996 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b70e54-f161-4a24-89d8-6a57c854de92 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:38.091424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b70e54-f161-4a24-89d8-6a57c854de92 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:38.093557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 190/360] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:37 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:38.094414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b70e54-f161-4a24-89d8-6a57c854de92 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:38.102358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-56d5b913-0951-4d47-ae22-c83a85904017 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:38.103565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-56d5b913-0951-4d47-ae22-c83a85904017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:38.117458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:38.188027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 171/361] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:38.188919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5b913-0951-4d47-ae22-c83a85904017 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:38.251270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b70e54-f161-4a24-89d8-6a57c854de92 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:38.294470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e5166-404b-4f30-bca9-c602b6ef7a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:38.295915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e5166-404b-4f30-bca9-c602b6ef7a94 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:38.296662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e5166-404b-4f30-bca9-c602b6ef7a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:38.297178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e5166-404b-4f30-bca9-c602b6ef7a94 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:38.298650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e5166-404b-4f30-bca9-c602b6ef7a94 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:38.299167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e5166-404b-4f30-bca9-c602b6ef7a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:38.300878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e5166-404b-4f30-bca9-c602b6ef7a94 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:38.310379 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:38.312124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 191/362] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:38.313187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e5166-404b-4f30-bca9-c602b6ef7a94 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:38.320463 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5b913-0951-4d47-ae22-c83a85904017 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:38.358210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da480f5-e54d-41e0-8d9d-8699194a0929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:38.358582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da480f5-e54d-41e0-8d9d-8699194a0929 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:38.358864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da480f5-e54d-41e0-8d9d-8699194a0929 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:38.359132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da480f5-e54d-41e0-8d9d-8699194a0929 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:38.359879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da480f5-e54d-41e0-8d9d-8699194a0929 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:38.360161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da480f5-e54d-41e0-8d9d-8699194a0929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:38.361534 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da480f5-e54d-41e0-8d9d-8699194a0929 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:38.370876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 172/363] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:38.371555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da480f5-e54d-41e0-8d9d-8699194a0929 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:38.415303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:38.444783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e5166-404b-4f30-bca9-c602b6ef7a94 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:38.486460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da480f5-e54d-41e0-8d9d-8699194a0929 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:38.487404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea693e3d-f9c9-4299-b510-223e0d8e6677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:38.488251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea693e3d-f9c9-4299-b510-223e0d8e6677 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:38.488726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea693e3d-f9c9-4299-b510-223e0d8e6677 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:38.489203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea693e3d-f9c9-4299-b510-223e0d8e6677 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:38.490540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea693e3d-f9c9-4299-b510-223e0d8e6677 None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:38.492172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea693e3d-f9c9-4299-b510-223e0d8e6677 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:38.494346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 192/364] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:38 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:38.495320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea693e3d-f9c9-4299-b510-223e0d8e6677 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:38.538199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a86ef-c986-4462-b892-8c54b1122f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:38.539012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a86ef-c986-4462-b892-8c54b1122f26 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:38.539291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a86ef-c986-4462-b892-8c54b1122f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:38.539623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a86ef-c986-4462-b892-8c54b1122f26 None admin] PATH_INFO: `/v3/groups/03e5023068a54913a7187468f0fb19bc` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:38.541761 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a86ef-c986-4462-b892-8c54b1122f26 None admin] RBAC: Authorizing `identity:get_group(group_id=03e5023068a54913a7187468f0fb19bc)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:38.543400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a86ef-c986-4462-b892-8c54b1122f26 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:38.545346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 173/365] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:38 2022] GET /identity/v3/groups/03e5023068a54913a7187468f0fb19bc => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:38.546799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a86ef-c986-4462-b892-8c54b1122f26 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:38.578506 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:38.640349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea693e3d-f9c9-4299-b510-223e0d8e6677 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:38.685788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a86ef-c986-4462-b892-8c54b1122f26 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:38.695780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d163c-f099-4229-8df5-1ef7066e8ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:38.696654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d163c-f099-4229-8df5-1ef7066e8ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:38.696932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d163c-f099-4229-8df5-1ef7066e8ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:38.697245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d163c-f099-4229-8df5-1ef7066e8ba9 None admin] PATH_INFO: `/v3/users/9387a2657405430591671b2eb24aaf3d` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:38.698436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d163c-f099-4229-8df5-1ef7066e8ba9 None admin] RBAC: Authorizing `identity:get_user(user_id=9387a2657405430591671b2eb24aaf3d)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:38.699865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d163c-f099-4229-8df5-1ef7066e8ba9 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:38.701894 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 193/366] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:24:38 2022] GET /identity/v3/users/9387a2657405430591671b2eb24aaf3d => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:38.702756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d163c-f099-4229-8df5-1ef7066e8ba9 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:38.730178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:38.734881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb6c71-f93c-4855-8cbf-580730eb917b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:38.736186 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb6c71-f93c-4855-8cbf-580730eb917b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:38.736469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb6c71-f93c-4855-8cbf-580730eb917b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:38.736794 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb6c71-f93c-4855-8cbf-580730eb917b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:38.830389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb6c71-f93c-4855-8cbf-580730eb917b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:38.832675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb6c71-f93c-4855-8cbf-580730eb917b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:38.837138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d163c-f099-4229-8df5-1ef7066e8ba9 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:38.866866 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:38.901627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f7c66-90f3-419a-8c7b-682a9e5ca0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:38.902270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f7c66-90f3-419a-8c7b-682a9e5ca0de None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:38.902550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f7c66-90f3-419a-8c7b-682a9e5ca0de None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:38.902822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f7c66-90f3-419a-8c7b-682a9e5ca0de None admin] PATH_INFO: `/v3/groups/03e5023068a54913a7187468f0fb19bc` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:38.904412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f7c66-90f3-419a-8c7b-682a9e5ca0de None admin] RBAC: Authorizing `identity:get_group(group_id=03e5023068a54913a7187468f0fb19bc)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:38.905878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f7c66-90f3-419a-8c7b-682a9e5ca0de None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:38.908257 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 194/367] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:38 2022] GET /identity/v3/groups/03e5023068a54913a7187468f0fb19bc => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:38.908967 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f7c66-90f3-419a-8c7b-682a9e5ca0de None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:38.960751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 174/368] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:38.961072 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb6c71-f93c-4855-8cbf-580730eb917b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:39.016161 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:39.041647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f7c66-90f3-419a-8c7b-682a9e5ca0de None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:39.084054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb6c71-f93c-4855-8cbf-580730eb917b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:39.088218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a8050-be32-49eb-8a63-110582e01d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:39.088850 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a8050-be32-49eb-8a63-110582e01d2b None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:39.089151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a8050-be32-49eb-8a63-110582e01d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:39.089425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a8050-be32-49eb-8a63-110582e01d2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:39.101700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a8050-be32-49eb-8a63-110582e01d2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-ca7a8050-be32-49eb-8a63-110582e01d2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:39.106574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:39.107652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:39.108708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 195/369] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:24:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:39.109556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a8050-be32-49eb-8a63-110582e01d2b None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:39.113146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:39.130243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ab861-478f-4fff-aeeb-99cc5ba717f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:39.130974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ab861-478f-4fff-aeeb-99cc5ba717f9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:39.131194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ab861-478f-4fff-aeeb-99cc5ba717f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:39.131474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ab861-478f-4fff-aeeb-99cc5ba717f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:39.132213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ab861-478f-4fff-aeeb-99cc5ba717f9 None admin] RBAC: Adding query filter params (group.id=03e5023068a54913a7187468f0fb19bc, role.id=89694ebde7824b8190bd5b15ece2b11e, scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:39.132524 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ab861-478f-4fff-aeeb-99cc5ba717f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:39.134014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ab861-478f-4fff-aeeb-99cc5ba717f9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:39.141496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 175/370] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 08:24:38 2022] GET /identity/v3/role_assignments?group.id=03e5023068a54913a7187468f0fb19bc&scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859&role.id=89694ebde7824b8190bd5b15ece2b11e => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:39.142745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ab861-478f-4fff-aeeb-99cc5ba717f9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:39.252595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a8050-be32-49eb-8a63-110582e01d2b None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:39.278763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ab861-478f-4fff-aeeb-99cc5ba717f9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:39.307563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659af7e-71f7-4375-b5c0-1865a1f02d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:39.308207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659af7e-71f7-4375-b5c0-1865a1f02d16 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:39.308491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659af7e-71f7-4375-b5c0-1865a1f02d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:39.308856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659af7e-71f7-4375-b5c0-1865a1f02d16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:39.309583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659af7e-71f7-4375-b5c0-1865a1f02d16 None admin] RBAC: Adding query filter params (role.id=718dd530c67f4a1891e292e729330f51, user.id=9387a2657405430591671b2eb24aaf3d, scope.system=all) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:39.310006 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659af7e-71f7-4375-b5c0-1865a1f02d16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:39.311578 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659af7e-71f7-4375-b5c0-1865a1f02d16 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:39.323089 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:39.326553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ea8a6-004f-487c-a2cb-9e2b02128988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:39.327485 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ea8a6-004f-487c-a2cb-9e2b02128988 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:39.327959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ea8a6-004f-487c-a2cb-9e2b02128988 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:39.328367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 196/371] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Sep 28 08:24:39 2022] GET /identity/v3/role_assignments?user.id=9387a2657405430591671b2eb24aaf3d&scope.system=all&role.id=718dd530c67f4a1891e292e729330f51 => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:39.328654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ea8a6-004f-487c-a2cb-9e2b02128988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:39.329809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659af7e-71f7-4375-b5c0-1865a1f02d16 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:39.330468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ea8a6-004f-487c-a2cb-9e2b02128988 None admin] RBAC: Adding query filter params (group.id=03e5023068a54913a7187468f0fb19bc, role.id=4c80373ee4c34171a7f29e5ab6c09ce0, scope.project.id=53c2a41061214641a61babeb674afeae) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:39.330999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ea8a6-004f-487c-a2cb-9e2b02128988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:39.332915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ea8a6-004f-487c-a2cb-9e2b02128988 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:39.341556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 176/372] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 08:24:39 2022] GET /identity/v3/role_assignments?group.id=03e5023068a54913a7187468f0fb19bc&scope.project.id=53c2a41061214641a61babeb674afeae&role.id=4c80373ee4c34171a7f29e5ab6c09ce0 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:39.343237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ea8a6-004f-487c-a2cb-9e2b02128988 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:39.412502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:39.471809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ea8a6-004f-487c-a2cb-9e2b02128988 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:39.491564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659af7e-71f7-4375-b5c0-1865a1f02d16 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:39.517655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205ed52-970d-4fad-be4e-5364c4756c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:39.518721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205ed52-970d-4fad-be4e-5364c4756c1d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:39.519197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205ed52-970d-4fad-be4e-5364c4756c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:39.519665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205ed52-970d-4fad-be4e-5364c4756c1d None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:39.520554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205ed52-970d-4fad-be4e-5364c4756c1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:39.521073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205ed52-970d-4fad-be4e-5364c4756c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:39.522717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205ed52-970d-4fad-be4e-5364c4756c1d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:39.542199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 177/373] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:39.549412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205ed52-970d-4fad-be4e-5364c4756c1d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:39.559276 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae0934-7b6c-4d83-9dca-9eb0e1e921ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:39.559948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae0934-7b6c-4d83-9dca-9eb0e1e921ec None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:39.560229 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae0934-7b6c-4d83-9dca-9eb0e1e921ec None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:39.560509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae0934-7b6c-4d83-9dca-9eb0e1e921ec None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:39.564279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae0934-7b6c-4d83-9dca-9eb0e1e921ec None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:39.565945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae0934-7b6c-4d83-9dca-9eb0e1e921ec None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:39.568167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 197/374] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:39 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:39.575870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae0934-7b6c-4d83-9dca-9eb0e1e921ec None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:39.679503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:39.740017 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae0934-7b6c-4d83-9dca-9eb0e1e921ec None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:39.775113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205ed52-970d-4fad-be4e-5364c4756c1d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:39.790409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ee5ed-e2ce-4ffa-bcdc-0c1b831f387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:39.791063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ee5ed-e2ce-4ffa-bcdc-0c1b831f387c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:39.791343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ee5ed-e2ce-4ffa-bcdc-0c1b831f387c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:39.791613 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ee5ed-e2ce-4ffa-bcdc-0c1b831f387c None admin] PATH_INFO: `/v3/groups/03e5023068a54913a7187468f0fb19bc` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:39.792761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ee5ed-e2ce-4ffa-bcdc-0c1b831f387c None admin] RBAC: Authorizing `identity:get_group(group_id=03e5023068a54913a7187468f0fb19bc)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:39.794104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:39.794293 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ee5ed-e2ce-4ffa-bcdc-0c1b831f387c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:39.798763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 198/375] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:39 2022] GET /identity/v3/groups/03e5023068a54913a7187468f0fb19bc => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:39.806158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ee5ed-e2ce-4ffa-bcdc-0c1b831f387c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:39.850295 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98b7d9-09d6-422e-a527-93c2f036029d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:39.850921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98b7d9-09d6-422e-a527-93c2f036029d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:39.851198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98b7d9-09d6-422e-a527-93c2f036029d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:39.851468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98b7d9-09d6-422e-a527-93c2f036029d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:39.991604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98b7d9-09d6-422e-a527-93c2f036029d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:39.993199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98b7d9-09d6-422e-a527-93c2f036029d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:40.034356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ee5ed-e2ce-4ffa-bcdc-0c1b831f387c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:40.090374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23462ecf-f92b-421b-a27a-58f123c112c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:40.091001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23462ecf-f92b-421b-a27a-58f123c112c3 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:40.091551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23462ecf-f92b-421b-a27a-58f123c112c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:40.091884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23462ecf-f92b-421b-a27a-58f123c112c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:40.092999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23462ecf-f92b-421b-a27a-58f123c112c3 None admin] RBAC: Adding query filter params (group.id=03e5023068a54913a7187468f0fb19bc, role.id=89694ebde7824b8190bd5b15ece2b11e, scope.project.id=53c2a41061214641a61babeb674afeae) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:40.093484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23462ecf-f92b-421b-a27a-58f123c112c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:40.095426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23462ecf-f92b-421b-a27a-58f123c112c3 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:40.099362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:40.106395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 199/376] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 08:24:39 2022] GET /identity/v3/role_assignments?group.id=03e5023068a54913a7187468f0fb19bc&scope.project.id=53c2a41061214641a61babeb674afeae&role.id=89694ebde7824b8190bd5b15ece2b11e => generated 260 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:40.107818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce31e8-e7d6-47bb-8c2a-32d8ac0293d3 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:40.108201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce31e8-e7d6-47bb-8c2a-32d8ac0293d3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:40.108572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce31e8-e7d6-47bb-8c2a-32d8ac0293d3 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:40.111648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 200/377] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:40.119335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ea2f2-2db1-461b-a8ee-41fc43b82d38 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:40.119643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ea2f2-2db1-461b-a8ee-41fc43b82d38 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:40.119886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ea2f2-2db1-461b-a8ee-41fc43b82d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:40.147078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-196ea2f2-2db1-461b-a8ee-41fc43b82d38 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:40.148326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-196ea2f2-2db1-461b-a8ee-41fc43b82d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:40.155936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 178/378] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:40.162019 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98b7d9-09d6-422e-a527-93c2f036029d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:40.190486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:40.248675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 201/379] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:40.345940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98b7d9-09d6-422e-a527-93c2f036029d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:40.399181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8980f-1675-43d8-85ed-757c464af31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:40.400195 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8980f-1675-43d8-85ed-757c464af31c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:40.400926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8980f-1675-43d8-85ed-757c464af31c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:40.401431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8980f-1675-43d8-85ed-757c464af31c None admin] PATH_INFO: `/v3/projects/c189a0d1d6b0443ba4af2cd3c46f5c43` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:40.402799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8980f-1675-43d8-85ed-757c464af31c None admin] RBAC: Authorizing `identity:get_project(project_id=c189a0d1d6b0443ba4af2cd3c46f5c43)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:40.404496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8980f-1675-43d8-85ed-757c464af31c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:40.406503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 179/380] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:40 2022] GET /identity/v3/projects/c189a0d1d6b0443ba4af2cd3c46f5c43 => generated 344 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:40.414587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ea2f2-2db1-461b-a8ee-41fc43b82d38 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:40.551791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ea2f2-2db1-461b-a8ee-41fc43b82d38 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:40.590276 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:40.597944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a5601-aa78-4c66-980c-398e2a3eb430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:40.598561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a5601-aa78-4c66-980c-398e2a3eb430 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:40.601443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a5601-aa78-4c66-980c-398e2a3eb430 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:40.601728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a5601-aa78-4c66-980c-398e2a3eb430 None admin] PATH_INFO: `/v3/groups/c5275cb1df554bf5849b537d30001c43` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:40.607028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a5601-aa78-4c66-980c-398e2a3eb430 None admin] RBAC: Authorizing `identity:get_group(group_id=c5275cb1df554bf5849b537d30001c43)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:40.608472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a5601-aa78-4c66-980c-398e2a3eb430 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:40.610510 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 202/381] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:40 2022] GET /identity/v3/groups/c5275cb1df554bf5849b537d30001c43 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:40.616269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8980f-1675-43d8-85ed-757c464af31c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:40.713259 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a5601-aa78-4c66-980c-398e2a3eb430 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:40.746172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:40.779587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8980f-1675-43d8-85ed-757c464af31c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:40.845366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9790eac-6ea0-416f-b158-61cd48d5bc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:40.845985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9790eac-6ea0-416f-b158-61cd48d5bc9b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:40.846262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9790eac-6ea0-416f-b158-61cd48d5bc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:40.846534 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9790eac-6ea0-416f-b158-61cd48d5bc9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:40.847265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9790eac-6ea0-416f-b158-61cd48d5bc9b None admin] RBAC: Adding query filter params (group.id=c5275cb1df554bf5849b537d30001c43, role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=c189a0d1d6b0443ba4af2cd3c46f5c43) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:40.847572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9790eac-6ea0-416f-b158-61cd48d5bc9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:40.849070 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9790eac-6ea0-416f-b158-61cd48d5bc9b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:40.864996 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 180/382] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 08:24:40 2022] GET /identity/v3/role_assignments?group.id=c5275cb1df554bf5849b537d30001c43&scope.project.id=c189a0d1d6b0443ba4af2cd3c46f5c43&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:40.920819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a5601-aa78-4c66-980c-398e2a3eb430 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:40.964783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12784518-d8db-412d-85c1-954bed8618ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:40.965720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12784518-d8db-412d-85c1-954bed8618ed None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:40.966172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12784518-d8db-412d-85c1-954bed8618ed None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:40.966629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12784518-d8db-412d-85c1-954bed8618ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:40.979533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12784518-d8db-412d-85c1-954bed8618ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-12784518-d8db-412d-85c1-954bed8618ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:40.984964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:40.986696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:40.987681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:40.987681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:40.987681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:40.987681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:40.987681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:40.987681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:40.987681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:40.987681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:40.987681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:40.987681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:40.987681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:40.987681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:40.990061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 203/383] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:40.996304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9790eac-6ea0-416f-b158-61cd48d5bc9b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:41.133674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9790eac-6ea0-416f-b158-61cd48d5bc9b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:41.176306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f102934-c42b-4e99-8dbf-943a48d4dfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:41.176932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f102934-c42b-4e99-8dbf-943a48d4dfb2 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:41.177240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f102934-c42b-4e99-8dbf-943a48d4dfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:41.177514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f102934-c42b-4e99-8dbf-943a48d4dfb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:41.178183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f102934-c42b-4e99-8dbf-943a48d4dfb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:41.178484 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f102934-c42b-4e99-8dbf-943a48d4dfb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:41.179864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f102934-c42b-4e99-8dbf-943a48d4dfb2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:41.189861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 181/384] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:41.195379 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12784518-d8db-412d-85c1-954bed8618ed None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:41.282800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:41.338474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12784518-d8db-412d-85c1-954bed8618ed None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:41.387594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776a222-a529-4cd7-9454-05e60dd44d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:41.388615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776a222-a529-4cd7-9454-05e60dd44d9d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:41.389127 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776a222-a529-4cd7-9454-05e60dd44d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:41.389630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776a222-a529-4cd7-9454-05e60dd44d9d None admin] PATH_INFO: `/v3/groups/03e5023068a54913a7187468f0fb19bc` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:41.390955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776a222-a529-4cd7-9454-05e60dd44d9d None admin] RBAC: Authorizing `identity:get_group(group_id=03e5023068a54913a7187468f0fb19bc)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:41.397193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776a222-a529-4cd7-9454-05e60dd44d9d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:41.399470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 204/385] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:41 2022] GET /identity/v3/groups/03e5023068a54913a7187468f0fb19bc => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:41.405066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f102934-c42b-4e99-8dbf-943a48d4dfb2 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:41.425640 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:41.563118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f102934-c42b-4e99-8dbf-943a48d4dfb2 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:41.621899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee326b8-c496-4cef-871f-a231b5c46003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:41.622610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee326b8-c496-4cef-871f-a231b5c46003 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:41.622926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee326b8-c496-4cef-871f-a231b5c46003 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:41.623202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee326b8-c496-4cef-871f-a231b5c46003 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:41.624493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee326b8-c496-4cef-871f-a231b5c46003 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:41.626632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee326b8-c496-4cef-871f-a231b5c46003 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:41.632469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 182/386] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:41 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:41.636215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776a222-a529-4cd7-9454-05e60dd44d9d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:41.775952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776a222-a529-4cd7-9454-05e60dd44d9d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:41.804379 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:41.818919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70334bf4-8d70-44af-82cd-183915e18bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:41.819627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70334bf4-8d70-44af-82cd-183915e18bbd None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:41.819954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70334bf4-8d70-44af-82cd-183915e18bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:41.820275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70334bf4-8d70-44af-82cd-183915e18bbd None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859/groups/03e5023068a54913a7187468f0fb19bc/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:41.822279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70334bf4-8d70-44af-82cd-183915e18bbd None admin] RBAC: Authorizing `identity:create_grant(project_id=c61d2a2c3629481f80a0cb5f9c63f859, group_id=03e5023068a54913a7187468f0fb19bc, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:41.823733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70334bf4-8d70-44af-82cd-183915e18bbd None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:41.827001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494b4bd-dbe8-49ef-a087-630264cec121 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:41.827288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494b4bd-dbe8-49ef-a087-630264cec121 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:41.827549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494b4bd-dbe8-49ef-a087-630264cec121 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:41.828924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 183/387] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:24:41.836004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047bb73-7834-455a-a5d6-6d5ee8157165 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:41.836715 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047bb73-7834-455a-a5d6-6d5ee8157165 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:41.837259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047bb73-7834-455a-a5d6-6d5ee8157165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:41.853139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 205/388] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 08:24:41 2022] PUT /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859/groups/03e5023068a54913a7187468f0fb19bc/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:41.879684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-d047bb73-7834-455a-a5d6-6d5ee8157165 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:41.881508 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d047bb73-7834-455a-a5d6-6d5ee8157165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:42.004356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 184/389] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:42.377206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d90cf-6056-4404-8f87-e4902c16be17 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:42.377951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d90cf-6056-4404-8f87-e4902c16be17 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:42.378451 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d90cf-6056-4404-8f87-e4902c16be17 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:42.380275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 206/390] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:42.386598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b314303-9492-468b-bbbd-f42e8ccd93c5 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:42.388103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b314303-9492-468b-bbbd-f42e8ccd93c5 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:42.388645 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b314303-9492-468b-bbbd-f42e8ccd93c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:42.408826 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:42.420356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d90cf-6056-4404-8f87-e4902c16be17 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:42.421468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-7b314303-9492-468b-bbbd-f42e8ccd93c5 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:42.422686 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7b314303-9492-468b-bbbd-f42e8ccd93c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:42.529351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 185/391] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:42.586162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d90cf-6056-4404-8f87-e4902c16be17 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:42.624287 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59cbe8-bc47-4737-8c46-6a2531ed8b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:42.625315 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59cbe8-bc47-4737-8c46-6a2531ed8b7d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:42.625830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59cbe8-bc47-4737-8c46-6a2531ed8b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:42.626866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59cbe8-bc47-4737-8c46-6a2531ed8b7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:42.638995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59cbe8-bc47-4737-8c46-6a2531ed8b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-4c59cbe8-bc47-4737-8c46-6a2531ed8b7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:42.645537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:42.646476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:42.647343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:42.647343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:42.647343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:42.647343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:42.647343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:42.647343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:42.647343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:42.647343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:42.647343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:42.647343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:42.647343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:42.647343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:42.650033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 207/392] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:24:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:42.656494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b314303-9492-468b-bbbd-f42e8ccd93c5 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:42.788053 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b314303-9492-468b-bbbd-f42e8ccd93c5 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:42.826412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e1edb-7c77-4ee0-870e-677d97f71345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:42.827772 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e1edb-7c77-4ee0-870e-677d97f71345 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:42.828276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e1edb-7c77-4ee0-870e-677d97f71345 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:42.828783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e1edb-7c77-4ee0-870e-677d97f71345 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:42.830219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e1edb-7c77-4ee0-870e-677d97f71345 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:42.830721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e1edb-7c77-4ee0-870e-677d97f71345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:42.832287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e1edb-7c77-4ee0-870e-677d97f71345 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:42.834913 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:42.842939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 186/393] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:42.849345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59cbe8-bc47-4737-8c46-6a2531ed8b7d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:42.898543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e1edb-7c77-4ee0-870e-677d97f71345 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:42.899979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:43.027681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59cbe8-bc47-4737-8c46-6a2531ed8b7d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:43.080125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e1edb-7c77-4ee0-870e-677d97f71345 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:43.106350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b33c7-c13e-4be0-9d58-7b3300a81ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:43.107050 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b33c7-c13e-4be0-9d58-7b3300a81ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:43.107339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b33c7-c13e-4be0-9d58-7b3300a81ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:43.107608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b33c7-c13e-4be0-9d58-7b3300a81ca5 None admin] PATH_INFO: `/v3/groups/03e5023068a54913a7187468f0fb19bc` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:43.113123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b33c7-c13e-4be0-9d58-7b3300a81ca5 None admin] RBAC: Authorizing `identity:get_group(group_id=03e5023068a54913a7187468f0fb19bc)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:43.117814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b33c7-c13e-4be0-9d58-7b3300a81ca5 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:43.119914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 208/394] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:42 2022] GET /identity/v3/groups/03e5023068a54913a7187468f0fb19bc => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:43.125811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b33c7-c13e-4be0-9d58-7b3300a81ca5 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:43.138490 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a92f68-ba37-40aa-8380-a9c03899b000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:43.139141 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a92f68-ba37-40aa-8380-a9c03899b000 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:43.139419 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a92f68-ba37-40aa-8380-a9c03899b000 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:43.139686 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a92f68-ba37-40aa-8380-a9c03899b000 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:43.177805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a92f68-ba37-40aa-8380-a9c03899b000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-20a92f68-ba37-40aa-8380-a9c03899b000 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:43.183038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:43.183990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:43.184778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 187/395] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:43.194058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a92f68-ba37-40aa-8380-a9c03899b000 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:43.242404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b33c7-c13e-4be0-9d58-7b3300a81ca5 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:43.273479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:43.287815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912276bc-2001-404a-a7e2-276ba99415fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:43.288574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:43.290295 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912276bc-2001-404a-a7e2-276ba99415fa None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:43.290675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912276bc-2001-404a-a7e2-276ba99415fa None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:43.291050 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912276bc-2001-404a-a7e2-276ba99415fa None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:43.292802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912276bc-2001-404a-a7e2-276ba99415fa None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:43.295421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912276bc-2001-404a-a7e2-276ba99415fa None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:43.297414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 209/396] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:43 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:43.303188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fc85e-ecca-4fc1-b0bd-8e542e8fecb6 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:43.303188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fc85e-ecca-4fc1-b0bd-8e542e8fecb6 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:43.303188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fc85e-ecca-4fc1-b0bd-8e542e8fecb6 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:43.304058 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 210/397] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:43.304595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fc85e-ecca-4fc1-b0bd-8e542e8fecb6 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:43.349283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a92f68-ba37-40aa-8380-a9c03899b000 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:43.391626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497be00e-4c8a-4999-8756-d8ca13113eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:43.392485 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497be00e-4c8a-4999-8756-d8ca13113eac None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:43.393015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497be00e-4c8a-4999-8756-d8ca13113eac None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:43.393590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497be00e-4c8a-4999-8756-d8ca13113eac None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:43.395208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497be00e-4c8a-4999-8756-d8ca13113eac None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:43.395681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497be00e-4c8a-4999-8756-d8ca13113eac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:43.397304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497be00e-4c8a-4999-8756-d8ca13113eac None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:43.413338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 188/398] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:43.416214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca017ec-711a-46e5-932d-3984cf447299 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:43.416756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca017ec-711a-46e5-932d-3984cf447299 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:43.417034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca017ec-711a-46e5-932d-3984cf447299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:43.441402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fc85e-ecca-4fc1-b0bd-8e542e8fecb6 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:43.442457 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-1ca017ec-711a-46e5-932d-3984cf447299 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:43.445049 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1ca017ec-711a-46e5-932d-3984cf447299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:43.484761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637ee82-dda8-455d-9559-4a561c27ac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:43.486414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ee82-dda8-455d-9559-4a561c27ac52 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:43.486932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ee82-dda8-455d-9559-4a561c27ac52 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:43.487435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637ee82-dda8-455d-9559-4a561c27ac52 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859/groups/03e5023068a54913a7187468f0fb19bc/roles/89694ebde7824b8190bd5b15ece2b11e` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:43.489998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637ee82-dda8-455d-9559-4a561c27ac52 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61d2a2c3629481f80a0cb5f9c63f859, group_id=03e5023068a54913a7187468f0fb19bc, role_id=89694ebde7824b8190bd5b15ece2b11e)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:43.491946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637ee82-dda8-455d-9559-4a561c27ac52 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:43.510693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 211/399] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 08:24:43 2022] PUT /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859/groups/03e5023068a54913a7187468f0fb19bc/roles/89694ebde7824b8190bd5b15ece2b11e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:43.511965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637ee82-dda8-455d-9559-4a561c27ac52 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:43.559576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 189/400] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:43.611174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:43.705603 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637ee82-dda8-455d-9559-4a561c27ac52 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:43.751929 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33a6aa-2410-491e-8716-aa4d9903e732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:43.752585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33a6aa-2410-491e-8716-aa4d9903e732 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:43.752862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33a6aa-2410-491e-8716-aa4d9903e732 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:43.753165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33a6aa-2410-491e-8716-aa4d9903e732 None admin] PATH_INFO: `/v3/groups/03e5023068a54913a7187468f0fb19bc` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:43.754353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33a6aa-2410-491e-8716-aa4d9903e732 None admin] RBAC: Authorizing `identity:get_group(group_id=03e5023068a54913a7187468f0fb19bc)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:43.755923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33a6aa-2410-491e-8716-aa4d9903e732 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:43.757919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 212/401] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:43 2022] GET /identity/v3/groups/03e5023068a54913a7187468f0fb19bc => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:43.763567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca017ec-711a-46e5-932d-3984cf447299 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:43.824272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:43.867611 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cef3e-1724-438c-be9f-409bed5e403c None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:43.868227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cef3e-1724-438c-be9f-409bed5e403c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:43.868701 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cef3e-1724-438c-be9f-409bed5e403c None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:43.871540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 213/402] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:24:43.875838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8e77c-c2f9-490f-a764-493a3f928f85 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:43.876490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8e77c-c2f9-490f-a764-493a3f928f85 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:43.876957 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8e77c-c2f9-490f-a764-493a3f928f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:43.894881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca017ec-711a-46e5-932d-3984cf447299 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:43.896152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e1c8e77c-c2f9-490f-a764-493a3f928f85 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:43.897624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e1c8e77c-c2f9-490f-a764-493a3f928f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:43.934190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e641e26-c8fa-4332-a7c3-75d0a360966e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:43.935188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e641e26-c8fa-4332-a7c3-75d0a360966e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:43.935577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e641e26-c8fa-4332-a7c3-75d0a360966e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:43.935957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e641e26-c8fa-4332-a7c3-75d0a360966e None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:43.937412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e641e26-c8fa-4332-a7c3-75d0a360966e None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:43.939828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e641e26-c8fa-4332-a7c3-75d0a360966e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:43.941865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 190/403] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:43 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:43.942849 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e641e26-c8fa-4332-a7c3-75d0a360966e None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:43.988834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 214/404] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:43.989673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8e77c-c2f9-490f-a764-493a3f928f85 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:44.055530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:44.066268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e641e26-c8fa-4332-a7c3-75d0a360966e None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:44.099822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:44.134108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09632d-3f70-4040-8fef-8cfcc5bb89c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:44.134760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09632d-3f70-4040-8fef-8cfcc5bb89c3 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:44.135039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09632d-3f70-4040-8fef-8cfcc5bb89c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:44.135312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09632d-3f70-4040-8fef-8cfcc5bb89c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:44.154494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09632d-3f70-4040-8fef-8cfcc5bb89c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-3f09632d-3f70-4040-8fef-8cfcc5bb89c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:44.160142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:44.161022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:44.162002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 191/405] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:24:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:44.168572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09632d-3f70-4040-8fef-8cfcc5bb89c3 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:44.198430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8e77c-c2f9-490f-a764-493a3f928f85 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:44.271808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ff1dc-97b7-4130-809d-19631584dd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:44.272399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ff1dc-97b7-4130-809d-19631584dd08 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:44.272681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ff1dc-97b7-4130-809d-19631584dd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:44.272958 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ff1dc-97b7-4130-809d-19631584dd08 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae/groups/03e5023068a54913a7187468f0fb19bc/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:44.277695 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ff1dc-97b7-4130-809d-19631584dd08 None admin] RBAC: Authorizing `identity:create_grant(project_id=53c2a41061214641a61babeb674afeae, group_id=03e5023068a54913a7187468f0fb19bc, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:44.279146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ff1dc-97b7-4130-809d-19631584dd08 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:44.297606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 215/406] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 08:24:43 2022] PUT /identity/v3/projects/53c2a41061214641a61babeb674afeae/groups/03e5023068a54913a7187468f0fb19bc/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:44.347975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ff1dc-97b7-4130-809d-19631584dd08 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:44.358716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09632d-3f70-4040-8fef-8cfcc5bb89c3 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:44.425236 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134f8a7-1240-4a33-a691-4e1dce0a33d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:44.425897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134f8a7-1240-4a33-a691-4e1dce0a33d2 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:44.426140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134f8a7-1240-4a33-a691-4e1dce0a33d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:44.426400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134f8a7-1240-4a33-a691-4e1dce0a33d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:44.427081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134f8a7-1240-4a33-a691-4e1dce0a33d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:44.427386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134f8a7-1240-4a33-a691-4e1dce0a33d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:44.428935 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134f8a7-1240-4a33-a691-4e1dce0a33d2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:44.445517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 192/407] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:44.452182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134f8a7-1240-4a33-a691-4e1dce0a33d2 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:44.536357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:44.546189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:44.555600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ff1dc-97b7-4130-809d-19631584dd08 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:44.604009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f0045-363a-48a7-9cdd-5471fc42d274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:44.605060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f0045-363a-48a7-9cdd-5471fc42d274 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:44.605590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f0045-363a-48a7-9cdd-5471fc42d274 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:44.606063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f0045-363a-48a7-9cdd-5471fc42d274 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:44.619505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f0045-363a-48a7-9cdd-5471fc42d274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-b37f0045-363a-48a7-9cdd-5471fc42d274 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:44.625321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:44.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:44.627659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:44.627659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:44.627659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:44.627659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:44.627659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:44.627659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:44.627659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:44.627659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:44.627659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:44.627659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:44.627659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:44.627659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:44.631447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 216/408] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:24:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:44.641193 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134f8a7-1240-4a33-a691-4e1dce0a33d2 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:44.643254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f0045-363a-48a7-9cdd-5471fc42d274 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:44.686275 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d20c4-fbec-4ea5-90b1-40deb0392979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:44.687187 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d20c4-fbec-4ea5-90b1-40deb0392979 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:44.687468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d20c4-fbec-4ea5-90b1-40deb0392979 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:44.687744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d20c4-fbec-4ea5-90b1-40deb0392979 None admin] PATH_INFO: `/v3/groups/03e5023068a54913a7187468f0fb19bc` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:44.689681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d20c4-fbec-4ea5-90b1-40deb0392979 None admin] RBAC: Authorizing `identity:get_group(group_id=03e5023068a54913a7187468f0fb19bc)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:44.691727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d20c4-fbec-4ea5-90b1-40deb0392979 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:44.693788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 193/409] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:44 2022] GET /identity/v3/groups/03e5023068a54913a7187468f0fb19bc => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:44.695766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3a79f-def1-42d0-9176-c8ba47bde944 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:44.696056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3a79f-def1-42d0-9176-c8ba47bde944 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:44.696331 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3a79f-def1-42d0-9176-c8ba47bde944 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:44.697813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 194/410] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:44.702005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3a79f-def1-42d0-9176-c8ba47bde944 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:44.783008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f0045-363a-48a7-9cdd-5471fc42d274 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:44.815676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3a79f-def1-42d0-9176-c8ba47bde944 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:44.825515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:44.828122 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1b3c9-0a3e-4d7d-b7e4-2226d61512a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:44.828775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1b3c9-0a3e-4d7d-b7e4-2226d61512a5 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:44.830240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1b3c9-0a3e-4d7d-b7e4-2226d61512a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:44.830525 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1b3c9-0a3e-4d7d-b7e4-2226d61512a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:44.831926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1b3c9-0a3e-4d7d-b7e4-2226d61512a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:44.832235 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1b3c9-0a3e-4d7d-b7e4-2226d61512a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:44.834499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1b3c9-0a3e-4d7d-b7e4-2226d61512a5 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:44.844366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 217/411] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:44.845853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86763e-d9b9-4df8-b8be-ae3c3d982593 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:44.846141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86763e-d9b9-4df8-b8be-ae3c3d982593 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:44.846415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86763e-d9b9-4df8-b8be-ae3c3d982593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:44.854703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dc156-9167-441e-8f05-7de3f47a3689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:44.855494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dc156-9167-441e-8f05-7de3f47a3689 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:44.855877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dc156-9167-441e-8f05-7de3f47a3689 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:44.856254 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dc156-9167-441e-8f05-7de3f47a3689 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:44.857511 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939dc156-9167-441e-8f05-7de3f47a3689 None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:44.859184 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939dc156-9167-441e-8f05-7de3f47a3689 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:44.861330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 195/412] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:44 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:44.862282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dc156-9167-441e-8f05-7de3f47a3689 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:44.867034 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-9d86763e-d9b9-4df8-b8be-ae3c3d982593 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:44.868230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9d86763e-d9b9-4df8-b8be-ae3c3d982593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:44.884268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:44.954402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 218/413] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:44.957743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86763e-d9b9-4df8-b8be-ae3c3d982593 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:44.990580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dc156-9167-441e-8f05-7de3f47a3689 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:45.058771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4b8fb-a5c9-4dd8-88a2-a3ad38cc48e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:45.059333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4b8fb-a5c9-4dd8-88a2-a3ad38cc48e7 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:45.059619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4b8fb-a5c9-4dd8-88a2-a3ad38cc48e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:45.059894 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4b8fb-a5c9-4dd8-88a2-a3ad38cc48e7 None admin] PATH_INFO: `/v3/groups/c5275cb1df554bf5849b537d30001c43` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:45.065299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4b8fb-a5c9-4dd8-88a2-a3ad38cc48e7 None admin] RBAC: Authorizing `identity:get_group(group_id=c5275cb1df554bf5849b537d30001c43)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:45.067902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4b8fb-a5c9-4dd8-88a2-a3ad38cc48e7 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:45.071463 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 196/414] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:44 2022] GET /identity/v3/groups/c5275cb1df554bf5849b537d30001c43 => generated 229 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:45.078521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4b8fb-a5c9-4dd8-88a2-a3ad38cc48e7 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:45.164470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86763e-d9b9-4df8-b8be-ae3c3d982593 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:45.165431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:45.234286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4b8fb-a5c9-4dd8-88a2-a3ad38cc48e7 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:45.251122 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a42d45-ce7a-49ac-93e6-bbfba3089f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:45.252098 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42d45-ce7a-49ac-93e6-bbfba3089f05 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:45.252576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42d45-ce7a-49ac-93e6-bbfba3089f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:45.253044 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42d45-ce7a-49ac-93e6-bbfba3089f05 None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae/groups/03e5023068a54913a7187468f0fb19bc/roles/89694ebde7824b8190bd5b15ece2b11e` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:45.258875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a42d45-ce7a-49ac-93e6-bbfba3089f05 None admin] RBAC: Authorizing `identity:create_grant(project_id=53c2a41061214641a61babeb674afeae, group_id=03e5023068a54913a7187468f0fb19bc, role_id=89694ebde7824b8190bd5b15ece2b11e)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:45.260520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a42d45-ce7a-49ac-93e6-bbfba3089f05 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:45.272712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c55b02-c0e2-4fd7-86b4-7813acabfab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:45.273494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c55b02-c0e2-4fd7-86b4-7813acabfab0 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:45.273783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c55b02-c0e2-4fd7-86b4-7813acabfab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:45.274102 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c55b02-c0e2-4fd7-86b4-7813acabfab0 None admin] PATH_INFO: `/v3/projects/c189a0d1d6b0443ba4af2cd3c46f5c43` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:45.275589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c55b02-c0e2-4fd7-86b4-7813acabfab0 None admin] RBAC: Authorizing `identity:get_project(project_id=c189a0d1d6b0443ba4af2cd3c46f5c43)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:45.277989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c55b02-c0e2-4fd7-86b4-7813acabfab0 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:45.280374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 197/415] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:45 2022] GET /identity/v3/projects/c189a0d1d6b0443ba4af2cd3c46f5c43 => generated 344 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:45.287302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c55b02-c0e2-4fd7-86b4-7813acabfab0 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:45.295291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 219/416] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 08:24:44 2022] PUT /identity/v3/projects/53c2a41061214641a61babeb674afeae/groups/03e5023068a54913a7187468f0fb19bc/roles/89694ebde7824b8190bd5b15ece2b11e => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:24:45.327805 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a42d45-ce7a-49ac-93e6-bbfba3089f05 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:45.341626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:45.474797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c55b02-c0e2-4fd7-86b4-7813acabfab0 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:45.491952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:45.528669 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54695c09-2462-4a25-9dc6-7765bf5c714c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:45.529188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54695c09-2462-4a25-9dc6-7765bf5c714c None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:45.529415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54695c09-2462-4a25-9dc6-7765bf5c714c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:45.529750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54695c09-2462-4a25-9dc6-7765bf5c714c None admin] PATH_INFO: `/v3/projects/c189a0d1d6b0443ba4af2cd3c46f5c43/groups/c5275cb1df554bf5849b537d30001c43/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:45.531704 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54695c09-2462-4a25-9dc6-7765bf5c714c None admin] RBAC: Authorizing `identity:create_grant(project_id=c189a0d1d6b0443ba4af2cd3c46f5c43, group_id=c5275cb1df554bf5849b537d30001c43, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:45.533232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54695c09-2462-4a25-9dc6-7765bf5c714c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:45.550679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 198/417] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 08:24:45 2022] PUT /identity/v3/projects/c189a0d1d6b0443ba4af2cd3c46f5c43/groups/c5275cb1df554bf5849b537d30001c43/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:24:45.567361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a42d45-ce7a-49ac-93e6-bbfba3089f05 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:45.614657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8815aa-abd5-420b-8b07-53e15046d49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:45.615221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8815aa-abd5-420b-8b07-53e15046d49e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:45.615501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8815aa-abd5-420b-8b07-53e15046d49e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:45.615774 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8815aa-abd5-420b-8b07-53e15046d49e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:45.637493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8815aa-abd5-420b-8b07-53e15046d49e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-fb8815aa-abd5-420b-8b07-53e15046d49e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:45.642824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:45.643777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:45.644565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 220/418] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:45.650647 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54695c09-2462-4a25-9dc6-7765bf5c714c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:45.792729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54695c09-2462-4a25-9dc6-7765bf5c714c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:45.834032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9765b-958f-4fa8-955e-39222d734547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:45.834651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9765b-958f-4fa8-955e-39222d734547 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:45.834929 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9765b-958f-4fa8-955e-39222d734547 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:45.835200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9765b-958f-4fa8-955e-39222d734547 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:45.835870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9765b-958f-4fa8-955e-39222d734547 None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:45.836173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9765b-958f-4fa8-955e-39222d734547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:45.837591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9765b-958f-4fa8-955e-39222d734547 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:45.847613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 199/419] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:45.853032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8815aa-abd5-420b-8b07-53e15046d49e None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:45.931098 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:45.972112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8815aa-abd5-420b-8b07-53e15046d49e None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:46.023237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2e530-79fe-4a61-a305-edf18d7c85a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:46.023873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2e530-79fe-4a61-a305-edf18d7c85a5 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:46.024151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2e530-79fe-4a61-a305-edf18d7c85a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:46.024422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2e530-79fe-4a61-a305-edf18d7c85a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:46.129670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2e530-79fe-4a61-a305-edf18d7c85a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:46.131148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2e530-79fe-4a61-a305-edf18d7c85a5 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:46.137407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff206ce6-3239-40b2-9df1-59051cdd0a62 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:46.137692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff206ce6-3239-40b2-9df1-59051cdd0a62 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:46.137968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff206ce6-3239-40b2-9df1-59051cdd0a62 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:46.139307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 200/420] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:24:46.146829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860aff91-bc21-41b6-9cd5-dc3d6f059168 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:46.147120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860aff91-bc21-41b6-9cd5-dc3d6f059168 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:46.147402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860aff91-bc21-41b6-9cd5-dc3d6f059168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:46.193296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-860aff91-bc21-41b6-9cd5-dc3d6f059168 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:46.194738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-860aff91-bc21-41b6-9cd5-dc3d6f059168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:46.243115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:46.244271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:46.307254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 221/421] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 08:24:46.312874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2e530-79fe-4a61-a305-edf18d7c85a5 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:46.322490 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 201/422] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:46.445003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2e530-79fe-4a61-a305-edf18d7c85a5 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:46.480262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2df43-625c-4f1f-8dfb-a5feae399e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:46.480890 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2df43-625c-4f1f-8dfb-a5feae399e42 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:46.481198 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2df43-625c-4f1f-8dfb-a5feae399e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:46.481470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2df43-625c-4f1f-8dfb-a5feae399e42 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:46.482594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2df43-625c-4f1f-8dfb-a5feae399e42 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:46.484021 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2df43-625c-4f1f-8dfb-a5feae399e42 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:46.485731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 222/423] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:46 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:46.491475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860aff91-bc21-41b6-9cd5-dc3d6f059168 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:46.617417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860aff91-bc21-41b6-9cd5-dc3d6f059168 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:46.656473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26e34a-840e-419d-8f55-c452562cb3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:46.657256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26e34a-840e-419d-8f55-c452562cb3a1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:46.658029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26e34a-840e-419d-8f55-c452562cb3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:46.658304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26e34a-840e-419d-8f55-c452562cb3a1 None admin] PATH_INFO: `/v3/groups/03e5023068a54913a7187468f0fb19bc` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:46.659622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26e34a-840e-419d-8f55-c452562cb3a1 None admin] RBAC: Authorizing `identity:get_group(group_id=03e5023068a54913a7187468f0fb19bc)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:46.661183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26e34a-840e-419d-8f55-c452562cb3a1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:46.663169 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 202/424] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:46 2022] GET /identity/v3/groups/03e5023068a54913a7187468f0fb19bc => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:46.669423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2df43-625c-4f1f-8dfb-a5feae399e42 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:46.672526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26e34a-840e-419d-8f55-c452562cb3a1 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:46.712615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:46.722945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:46.789814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2df43-625c-4f1f-8dfb-a5feae399e42 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:46.823471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26e34a-840e-419d-8f55-c452562cb3a1 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:46.828215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353304c-b12a-4647-801f-9a3c390887f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:46.828921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353304c-b12a-4647-801f-9a3c390887f8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:46.829815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353304c-b12a-4647-801f-9a3c390887f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:46.830158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353304c-b12a-4647-801f-9a3c390887f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:46.830953 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353304c-b12a-4647-801f-9a3c390887f8 None admin] RBAC: Adding query filter params (group.id=03e5023068a54913a7187468f0fb19bc, role.id=4c80373ee4c34171a7f29e5ab6c09ce0, scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:46.831661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353304c-b12a-4647-801f-9a3c390887f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:46.833179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353304c-b12a-4647-801f-9a3c390887f8 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:46.845377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 223/425] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 08:24:46 2022] GET /identity/v3/role_assignments?group.id=03e5023068a54913a7187468f0fb19bc&scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859&role.id=4c80373ee4c34171a7f29e5ab6c09ce0 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:46.845377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e07c0c-a6e5-49dc-8c23-c7ec267f6449 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:46.845377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e07c0c-a6e5-49dc-8c23-c7ec267f6449 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:46.845377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e07c0c-a6e5-49dc-8c23-c7ec267f6449 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:46.845377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 224/426] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:46.850195 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94453e-7613-41ec-96f3-b1a75564f979 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:46.850826 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94453e-7613-41ec-96f3-b1a75564f979 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:46.851311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94453e-7613-41ec-96f3-b1a75564f979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:46.882706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-1a94453e-7613-41ec-96f3-b1a75564f979 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:46.885800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1a94453e-7613-41ec-96f3-b1a75564f979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:46.898675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02a653-0ab1-4930-99ca-37e4d97f6dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:46.899810 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02a653-0ab1-4930-99ca-37e4d97f6dff None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:46.900336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02a653-0ab1-4930-99ca-37e4d97f6dff None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:46.901105 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02a653-0ab1-4930-99ca-37e4d97f6dff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:46.925353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02a653-0ab1-4930-99ca-37e4d97f6dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-1b02a653-0ab1-4930-99ca-37e4d97f6dff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:46.934512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:46.935471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:46.936259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:46.936259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:46.936259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:46.936259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:46.936259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:46.936259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:46.936259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:46.936259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:46.936259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:46.936259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:46.936259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:46.936259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:46.941406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 203/427] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:24:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:46.947523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02a653-0ab1-4930-99ca-37e4d97f6dff None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:47.047290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 225/428] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:47.088396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:47.098857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02a653-0ab1-4930-99ca-37e4d97f6dff None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:47.137392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdc784-bb62-426d-930e-2e5939cc9a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:47.138036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdc784-bb62-426d-930e-2e5939cc9a3c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:47.138314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdc784-bb62-426d-930e-2e5939cc9a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:47.138590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdc784-bb62-426d-930e-2e5939cc9a3c None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:47.139256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdc784-bb62-426d-930e-2e5939cc9a3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:47.139560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdc784-bb62-426d-930e-2e5939cc9a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:47.140924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdc784-bb62-426d-930e-2e5939cc9a3c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:47.158973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 204/429] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:47.164247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94453e-7613-41ec-96f3-b1a75564f979 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:47.198497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:47.276408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94453e-7613-41ec-96f3-b1a75564f979 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:47.318750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce49f3-d34c-44e0-b537-cb03a8501c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:47.319299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce49f3-d34c-44e0-b537-cb03a8501c12 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:47.319578 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce49f3-d34c-44e0-b537-cb03a8501c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:47.319850 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce49f3-d34c-44e0-b537-cb03a8501c12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:47.357815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdc784-bb62-426d-930e-2e5939cc9a3c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:47.416948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce49f3-d34c-44e0-b537-cb03a8501c12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:47.418449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce49f3-d34c-44e0-b537-cb03a8501c12 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:47.468688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:47.533114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdc784-bb62-426d-930e-2e5939cc9a3c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:47.570687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 226/430] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:47.575983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7a20c-41c4-40fe-820c-87c5a210b28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:47.576181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce49f3-d34c-44e0-b537-cb03a8501c12 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:47.576634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7a20c-41c4-40fe-820c-87c5a210b28a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:47.576915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7a20c-41c4-40fe-820c-87c5a210b28a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:47.577220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7a20c-41c4-40fe-820c-87c5a210b28a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:47.592275 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7a20c-41c4-40fe-820c-87c5a210b28a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-e3f7a20c-41c4-40fe-820c-87c5a210b28a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:47.600271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:47.601197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:47.602003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:47.602003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:47.602003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:47.602003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:47.602003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:47.602003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:47.602003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:47.602003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:47.602003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:47.602003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:47.602003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 08:24:47.602003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:47.602689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 205/431] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:47.607923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7a20c-41c4-40fe-820c-87c5a210b28a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:47.632203 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:47.712484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce49f3-d34c-44e0-b537-cb03a8501c12 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:47.732546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7a20c-41c4-40fe-820c-87c5a210b28a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:47.756730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:47.764350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df99766-d6f7-4367-baff-96e242b695f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:47.765070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df99766-d6f7-4367-baff-96e242b695f2 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:47.765916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df99766-d6f7-4367-baff-96e242b695f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:47.766192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df99766-d6f7-4367-baff-96e242b695f2 None admin] PATH_INFO: `/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:47.768060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df99766-d6f7-4367-baff-96e242b695f2 None admin] RBAC: Authorizing `identity:get_project(project_id=c61d2a2c3629481f80a0cb5f9c63f859)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:47.769833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df99766-d6f7-4367-baff-96e242b695f2 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:47.771890 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 227/432] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:47 2022] GET /identity/v3/projects/c61d2a2c3629481f80a0cb5f9c63f859 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:47.780647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df99766-d6f7-4367-baff-96e242b695f2 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:47.781214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b97d05-34db-48e7-8ed6-f00808bb051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:47.781932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b97d05-34db-48e7-8ed6-f00808bb051f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:47.782753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b97d05-34db-48e7-8ed6-f00808bb051f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:47.783027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b97d05-34db-48e7-8ed6-f00808bb051f None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:47.783700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b97d05-34db-48e7-8ed6-f00808bb051f None admin] RBAC: Adding query filter params (name=member) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:47.784003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b97d05-34db-48e7-8ed6-f00808bb051f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:47.785612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b97d05-34db-48e7-8ed6-f00808bb051f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:47.797152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 206/433] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:24:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:47.804243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b97d05-34db-48e7-8ed6-f00808bb051f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:47.909218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df99766-d6f7-4367-baff-96e242b695f2 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:47.912618 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:47.945443 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b97d05-34db-48e7-8ed6-f00808bb051f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:47.956575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b86e2-477e-41f3-b409-31ae5a99224d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:47.957631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b86e2-477e-41f3-b409-31ae5a99224d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:47.958057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b86e2-477e-41f3-b409-31ae5a99224d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:47.958502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b86e2-477e-41f3-b409-31ae5a99224d None admin] PATH_INFO: `/v3/groups/03e5023068a54913a7187468f0fb19bc` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:47.959901 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b86e2-477e-41f3-b409-31ae5a99224d None admin] RBAC: Authorizing `identity:get_group(group_id=03e5023068a54913a7187468f0fb19bc)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:47.961751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b86e2-477e-41f3-b409-31ae5a99224d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:47.968420 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 228/434] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:47 2022] GET /identity/v3/groups/03e5023068a54913a7187468f0fb19bc => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:47.970128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b86e2-477e-41f3-b409-31ae5a99224d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:47.988388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb615bcd-d157-4e3f-8e3c-a19655753a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:47.989163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb615bcd-d157-4e3f-8e3c-a19655753a68 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:47.989578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb615bcd-d157-4e3f-8e3c-a19655753a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:47.989864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb615bcd-d157-4e3f-8e3c-a19655753a68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:48.049067 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:48.087566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b86e2-477e-41f3-b409-31ae5a99224d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:48.098076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb615bcd-d157-4e3f-8e3c-a19655753a68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:48.100340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb615bcd-d157-4e3f-8e3c-a19655753a68 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:48.126235 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb3d9a-cda2-47fd-b05b-f64facfda75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:48.127141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb3d9a-cda2-47fd-b05b-f64facfda75f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:48.127628 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb3d9a-cda2-47fd-b05b-f64facfda75f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:48.128092 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb3d9a-cda2-47fd-b05b-f64facfda75f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:48.128830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb3d9a-cda2-47fd-b05b-f64facfda75f None admin] RBAC: Adding query filter params (group.id=03e5023068a54913a7187468f0fb19bc, role.id=89694ebde7824b8190bd5b15ece2b11e, scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:48.129185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb3d9a-cda2-47fd-b05b-f64facfda75f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:48.130874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb3d9a-cda2-47fd-b05b-f64facfda75f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:48.138181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 229/435] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 08:24:47 2022] GET /identity/v3/role_assignments?group.id=03e5023068a54913a7187468f0fb19bc&scope.project.id=c61d2a2c3629481f80a0cb5f9c63f859&role.id=89694ebde7824b8190bd5b15ece2b11e => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:48.139717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c07be7-0d38-4d39-a19e-bf1407071c44 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:48.140185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c07be7-0d38-4d39-a19e-bf1407071c44 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:48.140712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c07be7-0d38-4d39-a19e-bf1407071c44 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:48.142763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 230/436] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:48.150356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdc434-62ff-486c-9e00-ad5d4f506ca9 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:48.151253 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdc434-62ff-486c-9e00-ad5d4f506ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:48.151736 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdc434-62ff-486c-9e00-ad5d4f506ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:48.172104 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-55bdc434-62ff-486c-9e00-ad5d4f506ca9 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:48.176641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-55bdc434-62ff-486c-9e00-ad5d4f506ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:48.207542 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:48.245002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 207/437] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:48.250638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb615bcd-d157-4e3f-8e3c-a19655753a68 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:48.273130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 231/438] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:48.370737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb615bcd-d157-4e3f-8e3c-a19655753a68 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:48.408192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998a775-8d62-4111-9cc3-aa4bb775e66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:48.408893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998a775-8d62-4111-9cc3-aa4bb775e66c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:48.409210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998a775-8d62-4111-9cc3-aa4bb775e66c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:48.409487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998a775-8d62-4111-9cc3-aa4bb775e66c None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:48.410636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998a775-8d62-4111-9cc3-aa4bb775e66c None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:48.412089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998a775-8d62-4111-9cc3-aa4bb775e66c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:48.414110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 208/439] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:48 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:48.419272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bdc434-62ff-486c-9e00-ad5d4f506ca9 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:48.452293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aab92c-a0ba-48df-bbe1-7675a9adb8e2 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:48.452498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aab92c-a0ba-48df-bbe1-7675a9adb8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:48.452767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aab92c-a0ba-48df-bbe1-7675a9adb8e2 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:48.454960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 209/440] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:24:48.460957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c3cb9-748d-4b16-9b1a-b3ff76e07d0d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:48.461262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c3cb9-748d-4b16-9b1a-b3ff76e07d0d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:48.461544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c3cb9-748d-4b16-9b1a-b3ff76e07d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:48.482901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-f60c3cb9-748d-4b16-9b1a-b3ff76e07d0d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:48.484144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f60c3cb9-748d-4b16-9b1a-b3ff76e07d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:48.490950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:48.550123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:48.556526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bdc434-62ff-486c-9e00-ad5d4f506ca9 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:48.589163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 210/441] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:48.589163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c3cb9-748d-4b16-9b1a-b3ff76e07d0d None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:48.599405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc9685-7011-444a-93cf-db7f2606330f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:48.600349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc9685-7011-444a-93cf-db7f2606330f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:48.600632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc9685-7011-444a-93cf-db7f2606330f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:48.600910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc9685-7011-444a-93cf-db7f2606330f None admin] PATH_INFO: `/v3/groups/03e5023068a54913a7187468f0fb19bc` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:48.602157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc9685-7011-444a-93cf-db7f2606330f None admin] RBAC: Authorizing `identity:get_group(group_id=03e5023068a54913a7187468f0fb19bc)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:48.603594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc9685-7011-444a-93cf-db7f2606330f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:48.605729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 232/442] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:48 2022] GET /identity/v3/groups/03e5023068a54913a7187468f0fb19bc => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:48.611802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc9685-7011-444a-93cf-db7f2606330f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:48.703761 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c3cb9-748d-4b16-9b1a-b3ff76e07d0d None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:48.728702 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc9685-7011-444a-93cf-db7f2606330f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:48.746906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a4539-a89e-4487-8f5d-6ce2bb18b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:48.747780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a4539-a89e-4487-8f5d-6ce2bb18b048 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:48.748201 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a4539-a89e-4487-8f5d-6ce2bb18b048 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:48.748644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a4539-a89e-4487-8f5d-6ce2bb18b048 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:48.761043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:48.765921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a4539-a89e-4487-8f5d-6ce2bb18b048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:48.769746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128484b9-8a4c-4d80-8ba2-1e127d2416de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:48.770381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128484b9-8a4c-4d80-8ba2-1e127d2416de None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:48.770661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128484b9-8a4c-4d80-8ba2-1e127d2416de None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:48.770934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128484b9-8a4c-4d80-8ba2-1e127d2416de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-625a4539-a89e-4487-8f5d-6ce2bb18b048 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:48.771029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:48.771870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:48.772659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:48.772659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:48.772659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:48.772659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:48.772659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:48.772659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:48.772659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:48.772659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:48.772659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:48.772659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:48.772659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:48.772659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:48.773286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128484b9-8a4c-4d80-8ba2-1e127d2416de None admin] RBAC: Adding query filter params (group.id=03e5023068a54913a7187468f0fb19bc, role.id=4c80373ee4c34171a7f29e5ab6c09ce0, scope.project.id=53c2a41061214641a61babeb674afeae) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:48.773286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128484b9-8a4c-4d80-8ba2-1e127d2416de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:48.773428 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 211/443] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:24:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:48.773495 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128484b9-8a4c-4d80-8ba2-1e127d2416de None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:48.778053 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a4539-a89e-4487-8f5d-6ce2bb18b048 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:48.789362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 233/444] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 08:24:48 2022] GET /identity/v3/role_assignments?group.id=03e5023068a54913a7187468f0fb19bc&scope.project.id=53c2a41061214641a61babeb674afeae&role.id=4c80373ee4c34171a7f29e5ab6c09ce0 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:48.820343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:48.876664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128484b9-8a4c-4d80-8ba2-1e127d2416de None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:48.897859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a4539-a89e-4487-8f5d-6ce2bb18b048 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:48.938384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfd1ea-d7da-4fe7-82bc-8cffb3bd746a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:48.939504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfd1ea-d7da-4fe7-82bc-8cffb3bd746a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:48.939783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfd1ea-d7da-4fe7-82bc-8cffb3bd746a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:48.940054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfd1ea-d7da-4fe7-82bc-8cffb3bd746a None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:48.940731 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfd1ea-d7da-4fe7-82bc-8cffb3bd746a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:48.941034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfd1ea-d7da-4fe7-82bc-8cffb3bd746a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:48.942497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfd1ea-d7da-4fe7-82bc-8cffb3bd746a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:48.957538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 212/445] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:48.957767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfd1ea-d7da-4fe7-82bc-8cffb3bd746a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:48.997123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128484b9-8a4c-4d80-8ba2-1e127d2416de None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:49.039465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295f776-aee5-44a1-b842-6ccd26f18a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:49.040347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295f776-aee5-44a1-b842-6ccd26f18a45 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:49.040803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295f776-aee5-44a1-b842-6ccd26f18a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:49.041079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295f776-aee5-44a1-b842-6ccd26f18a45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:49.059825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295f776-aee5-44a1-b842-6ccd26f18a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-7295f776-aee5-44a1-b842-6ccd26f18a45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:49.066807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:49.067852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:49.069943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 234/446] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:24:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:49.073928 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295f776-aee5-44a1-b842-6ccd26f18a45 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:49.083716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:49.090942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfd1ea-d7da-4fe7-82bc-8cffb3bd746a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:49.134698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9fc76-d5f1-459f-bfa9-227f0d503aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:49.135321 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9fc76-d5f1-459f-bfa9-227f0d503aab None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:49.135603 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9fc76-d5f1-459f-bfa9-227f0d503aab None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:49.135879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9fc76-d5f1-459f-bfa9-227f0d503aab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:49.148853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:49.207423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295f776-aee5-44a1-b842-6ccd26f18a45 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:49.214228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9fc76-d5f1-459f-bfa9-227f0d503aab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:49.215622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9fc76-d5f1-459f-bfa9-227f0d503aab None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:49.241720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca73ed4-e4cb-44f2-a5eb-9e515b397bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:49.242330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca73ed4-e4cb-44f2-a5eb-9e515b397bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:49.242606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca73ed4-e4cb-44f2-a5eb-9e515b397bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:49.242877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca73ed4-e4cb-44f2-a5eb-9e515b397bc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:49.243543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca73ed4-e4cb-44f2-a5eb-9e515b397bc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:49.243854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca73ed4-e4cb-44f2-a5eb-9e515b397bc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:49.245260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca73ed4-e4cb-44f2-a5eb-9e515b397bc8 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:49.253855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 235/447] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:49.259612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca73ed4-e4cb-44f2-a5eb-9e515b397bc8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:49.312334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:49.321332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 213/448] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 08:24:49.327123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9fc76-d5f1-459f-bfa9-227f0d503aab None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:49.362737 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca73ed4-e4cb-44f2-a5eb-9e515b397bc8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:49.376529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:49.398164 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba5745-1388-4a08-804c-da02a2e4dc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:49.398765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba5745-1388-4a08-804c-da02a2e4dc2c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:49.399041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba5745-1388-4a08-804c-da02a2e4dc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:49.399318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba5745-1388-4a08-804c-da02a2e4dc2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:49.429903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9fc76-d5f1-459f-bfa9-227f0d503aab None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:49.464899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee790a1-8882-49ff-9e10-beae202ae28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:49.465501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee790a1-8882-49ff-9e10-beae202ae28b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:49.465773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee790a1-8882-49ff-9e10-beae202ae28b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:49.466039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee790a1-8882-49ff-9e10-beae202ae28b None admin] PATH_INFO: `/v3/projects/53c2a41061214641a61babeb674afeae` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:49.467137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee790a1-8882-49ff-9e10-beae202ae28b None admin] RBAC: Authorizing `identity:get_project(project_id=53c2a41061214641a61babeb674afeae)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:49.467366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba5745-1388-4a08-804c-da02a2e4dc2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:49.468563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee790a1-8882-49ff-9e10-beae202ae28b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:49.468789 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba5745-1388-4a08-804c-da02a2e4dc2c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:49.470233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 214/449] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:49 2022] GET /identity/v3/projects/53c2a41061214641a61babeb674afeae => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:49.475793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee790a1-8882-49ff-9e10-beae202ae28b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:49.540015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:49.570180 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 236/450] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 08:24:49.576937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba5745-1388-4a08-804c-da02a2e4dc2c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:49.580570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee790a1-8882-49ff-9e10-beae202ae28b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:49.592679 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:49.620013 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7623b63-ca1f-42e1-999d-c7bcf26ea222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:49.621066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7623b63-ca1f-42e1-999d-c7bcf26ea222 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:49.621379 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7623b63-ca1f-42e1-999d-c7bcf26ea222 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:49.621657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7623b63-ca1f-42e1-999d-c7bcf26ea222 None admin] PATH_INFO: `/v3/groups/03e5023068a54913a7187468f0fb19bc` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:49.623111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7623b63-ca1f-42e1-999d-c7bcf26ea222 None admin] RBAC: Authorizing `identity:get_group(group_id=03e5023068a54913a7187468f0fb19bc)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:49.624531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7623b63-ca1f-42e1-999d-c7bcf26ea222 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:49.626906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 215/451] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:49 2022] GET /identity/v3/groups/03e5023068a54913a7187468f0fb19bc => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:49.632493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7623b63-ca1f-42e1-999d-c7bcf26ea222 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:49.681939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba5745-1388-4a08-804c-da02a2e4dc2c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:49.716295 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36beb5-6a9e-4ca3-b7bc-9dd83b5efb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:49.716838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36beb5-6a9e-4ca3-b7bc-9dd83b5efb8c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:49.717157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36beb5-6a9e-4ca3-b7bc-9dd83b5efb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:49.717439 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36beb5-6a9e-4ca3-b7bc-9dd83b5efb8c None admin] PATH_INFO: `/v3/projects/c189a0d1d6b0443ba4af2cd3c46f5c43` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:49.718594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36beb5-6a9e-4ca3-b7bc-9dd83b5efb8c None admin] RBAC: Authorizing `identity:get_project(project_id=c189a0d1d6b0443ba4af2cd3c46f5c43)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:49.720642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36beb5-6a9e-4ca3-b7bc-9dd83b5efb8c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:49.722316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 237/452] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:24:49 2022] GET /identity/v3/projects/c189a0d1d6b0443ba4af2cd3c46f5c43 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:49.727700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36beb5-6a9e-4ca3-b7bc-9dd83b5efb8c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:49.733256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7623b63-ca1f-42e1-999d-c7bcf26ea222 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:49.762608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:49.768481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a157e3-5566-4589-b366-4f6b848c7130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:49.769304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a157e3-5566-4589-b366-4f6b848c7130 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:49.769579 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a157e3-5566-4589-b366-4f6b848c7130 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:49.769849 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a157e3-5566-4589-b366-4f6b848c7130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:49.770574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a157e3-5566-4589-b366-4f6b848c7130 None admin] RBAC: Adding query filter params (group.id=03e5023068a54913a7187468f0fb19bc, role.id=89694ebde7824b8190bd5b15ece2b11e, scope.project.id=53c2a41061214641a61babeb674afeae) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:49.770877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a157e3-5566-4589-b366-4f6b848c7130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:49.772298 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a157e3-5566-4589-b366-4f6b848c7130 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:49.778422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 216/453] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 08:24:49 2022] GET /identity/v3/role_assignments?group.id=03e5023068a54913a7187468f0fb19bc&scope.project.id=53c2a41061214641a61babeb674afeae&role.id=89694ebde7824b8190bd5b15ece2b11e => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:49.811073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:49.832778 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36beb5-6a9e-4ca3-b7bc-9dd83b5efb8c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:49.870672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c803e4-fb12-4003-ad24-a41de6649186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:49.871314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c803e4-fb12-4003-ad24-a41de6649186 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:49.871589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c803e4-fb12-4003-ad24-a41de6649186 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:49.871858 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c803e4-fb12-4003-ad24-a41de6649186 None admin] PATH_INFO: `/v3/groups/c5275cb1df554bf5849b537d30001c43` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:49.873032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c803e4-fb12-4003-ad24-a41de6649186 None admin] RBAC: Authorizing `identity:get_group(group_id=c5275cb1df554bf5849b537d30001c43)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:49.874506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c803e4-fb12-4003-ad24-a41de6649186 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:49.876144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 238/454] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 08:24:49 2022] GET /identity/v3/groups/c5275cb1df554bf5849b537d30001c43 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:49.882173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a157e3-5566-4589-b366-4f6b848c7130 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:49.987851 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a157e3-5566-4589-b366-4f6b848c7130 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:50.021934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289847c3-0ad7-4348-a41a-792e96530813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:50.022536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289847c3-0ad7-4348-a41a-792e96530813 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:50.022811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289847c3-0ad7-4348-a41a-792e96530813 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:50.023075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289847c3-0ad7-4348-a41a-792e96530813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:50.023792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289847c3-0ad7-4348-a41a-792e96530813 None admin] RBAC: Adding query filter params (group.id=c5275cb1df554bf5849b537d30001c43, role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=c189a0d1d6b0443ba4af2cd3c46f5c43) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:50.024094 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289847c3-0ad7-4348-a41a-792e96530813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:50.025543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289847c3-0ad7-4348-a41a-792e96530813 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:50.031774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 217/455] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 08:24:49 2022] GET /identity/v3/role_assignments?group.id=c5275cb1df554bf5849b537d30001c43&scope.project.id=c189a0d1d6b0443ba4af2cd3c46f5c43&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:52.598668 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721331f-e063-4241-8009-256094915ef1 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:52.599174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721331f-e063-4241-8009-256094915ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:52.599248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721331f-e063-4241-8009-256094915ef1 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:52.600775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 239/456] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:52.605818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bafb9d-a400-497b-b292-14b09fbc8029 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:52.606618 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bafb9d-a400-497b-b292-14b09fbc8029 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:52.607099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bafb9d-a400-497b-b292-14b09fbc8029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:52.630359 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-69bafb9d-a400-497b-b292-14b09fbc8029 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:52.631906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-69bafb9d-a400-497b-b292-14b09fbc8029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:52.708435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:52.722785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 218/457] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:53.143692 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721331f-e063-4241-8009-256094915ef1 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:53.171183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b372e-d3f3-4726-b9b8-5aa6e0fb7570 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:53.171811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b372e-d3f3-4726-b9b8-5aa6e0fb7570 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:53.172279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b372e-d3f3-4726-b9b8-5aa6e0fb7570 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:53.174167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 219/458] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:53.179091 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3892-0b2b-4416-bdb0-8af448d3c995 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:53.179625 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3892-0b2b-4416-bdb0-8af448d3c995 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:53.180093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3892-0b2b-4416-bdb0-8af448d3c995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:53.208533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-06ea3892-0b2b-4416-bdb0-8af448d3c995 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:53.209756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-06ea3892-0b2b-4416-bdb0-8af448d3c995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:53.295419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721331f-e063-4241-8009-256094915ef1 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:53.305117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 220/459] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:53.305117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a891b0-eb1a-4c42-b0f5-1e2f88b6983f None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:53.305117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a891b0-eb1a-4c42-b0f5-1e2f88b6983f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:53.305117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a891b0-eb1a-4c42-b0f5-1e2f88b6983f None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:53.305117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 221/460] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:53.318594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f6ab2-bb3d-4861-81b5-d8104de79919 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:53.318876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f6ab2-bb3d-4861-81b5-d8104de79919 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:53.319152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f6ab2-bb3d-4861-81b5-d8104de79919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:53.350178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-2e5f6ab2-bb3d-4861-81b5-d8104de79919 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:53.351368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2e5f6ab2-bb3d-4861-81b5-d8104de79919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:53.367712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca823c2-de0e-496b-a933-0abd349c97d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:53.368353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca823c2-de0e-496b-a933-0abd349c97d8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:53.368627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca823c2-de0e-496b-a933-0abd349c97d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:53.368898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca823c2-de0e-496b-a933-0abd349c97d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:53.384375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca823c2-de0e-496b-a933-0abd349c97d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:53.386677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca823c2-de0e-496b-a933-0abd349c97d8 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-1ca823c2-de0e-496b-a933-0abd349c97d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:24:53.404904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:53.405872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:24:53.406703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:24:53.406703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:53.406703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:53.406703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 240/461] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:53.407949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32508c3-69f6-41aa-8487-887c2538c5b5 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:53.408143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32508c3-69f6-41aa-8487-887c2538c5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:53.408651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32508c3-69f6-41aa-8487-887c2538c5b5 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:53.409881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 241/462] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:53.412216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32508c3-69f6-41aa-8487-887c2538c5b5 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:53.416008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:53.456787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 222/463] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 28 08:24:53.458082 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4357290-67d0-4045-8b94-d1a56bb42d8b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:53.458367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4357290-67d0-4045-8b94-d1a56bb42d8b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:53.458648 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4357290-67d0-4045-8b94-d1a56bb42d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:53.484084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-b4357290-67d0-4045-8b94-d1a56bb42d8b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:53.485628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b4357290-67d0-4045-8b94-d1a56bb42d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:53.550795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32508c3-69f6-41aa-8487-887c2538c5b5 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:53.577487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:53.615567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 223/464] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:53.623088 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e199a-b3c2-4b43-b859-e441a50ddbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:53.623715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e199a-b3c2-4b43-b859-e441a50ddbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:53.624000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e199a-b3c2-4b43-b859-e441a50ddbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:53.624279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e199a-b3c2-4b43-b859-e441a50ddbf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:53.624935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e199a-b3c2-4b43-b859-e441a50ddbf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:53.625272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e199a-b3c2-4b43-b859-e441a50ddbf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:53.627197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e199a-b3c2-4b43-b859-e441a50ddbf7 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:53.644615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 242/465] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:53.649901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4357290-67d0-4045-8b94-d1a56bb42d8b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:53.659712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e199a-b3c2-4b43-b859-e441a50ddbf7 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:53.779126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:53.824625 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4357290-67d0-4045-8b94-d1a56bb42d8b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:53.827245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e199a-b3c2-4b43-b859-e441a50ddbf7 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:53.863630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf936e-b247-4886-a684-8bbff8c8c5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:53.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf936e-b247-4886-a684-8bbff8c8c5ba None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:53.865621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf936e-b247-4886-a684-8bbff8c8c5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:53.866100 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf936e-b247-4886-a684-8bbff8c8c5ba None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:53.866954 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf936e-b247-4886-a684-8bbff8c8c5ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:53.868922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf936e-b247-4886-a684-8bbff8c8c5ba None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:53.875270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865b13a-64ec-4d6f-a9cf-79ff98a27fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:53.876430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865b13a-64ec-4d6f-a9cf-79ff98a27fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:53.876712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865b13a-64ec-4d6f-a9cf-79ff98a27fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:53.876987 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865b13a-64ec-4d6f-a9cf-79ff98a27fb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:53.893192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865b13a-64ec-4d6f-a9cf-79ff98a27fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:53.895051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865b13a-64ec-4d6f-a9cf-79ff98a27fb5 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:53.899660 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 224/466] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:24:53 2022] POST /identity/v3/users => generated 246 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:24:53.901170 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf936e-b247-4886-a684-8bbff8c8c5ba None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-8865b13a-64ec-4d6f-a9cf-79ff98a27fb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:24:53.913803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:53.914750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:24:53.916228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:24:53.916228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:53.916228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:53.916228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 243/467] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:53.926435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865b13a-64ec-4d6f-a9cf-79ff98a27fb5 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:53.970276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:54.053665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf936e-b247-4886-a684-8bbff8c8c5ba None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:54.074375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865b13a-64ec-4d6f-a9cf-79ff98a27fb5 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:54.093408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:54.097014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2fb00-8b73-4fc9-9474-494306a06f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:54.098341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2fb00-8b73-4fc9-9474-494306a06f5d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:54.098765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2fb00-8b73-4fc9-9474-494306a06f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:54.099178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2fb00-8b73-4fc9-9474-494306a06f5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:54.114495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2fb00-8b73-4fc9-9474-494306a06f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:54.116162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2fb00-8b73-4fc9-9474-494306a06f5d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:54.116507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6dd512-50dc-45b1-a1a5-7dbe6c79469a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:54.117177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6dd512-50dc-45b1-a1a5-7dbe6c79469a None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:54.117462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6dd512-50dc-45b1-a1a5-7dbe6c79469a None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:54.117737 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6dd512-50dc-45b1-a1a5-7dbe6c79469a None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:54.118385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6dd512-50dc-45b1-a1a5-7dbe6c79469a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:54.118689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6dd512-50dc-45b1-a1a5-7dbe6c79469a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:54.120046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6dd512-50dc-45b1-a1a5-7dbe6c79469a None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-02b2fb00-8b73-4fc9-9474-494306a06f5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:24:54.136086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:54.137284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:24:54.138644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:24:54.138644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:54.138644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:54.138834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 244/468] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:54.138906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 225/469] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:54.139119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6dd512-50dc-45b1-a1a5-7dbe6c79469a None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:54.143632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2fb00-8b73-4fc9-9474-494306a06f5d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:54.261404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:54.278714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2fb00-8b73-4fc9-9474-494306a06f5d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:54.290833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6dd512-50dc-45b1-a1a5-7dbe6c79469a None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:54.320281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70268f-ee14-455d-bef3-33f052f83b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:54.321345 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70268f-ee14-455d-bef3-33f052f83b7f None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:54.321817 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70268f-ee14-455d-bef3-33f052f83b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:54.322260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70268f-ee14-455d-bef3-33f052f83b7f None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:54.323168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70268f-ee14-455d-bef3-33f052f83b7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:54.324892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d70268f-ee14-455d-bef3-33f052f83b7f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:54.328602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f6897-718f-4667-9661-e780f3087c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:54.329812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f6897-718f-4667-9661-e780f3087c12 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:54.330086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f6897-718f-4667-9661-e780f3087c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:54.330358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f6897-718f-4667-9661-e780f3087c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:54.342724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f6897-718f-4667-9661-e780f3087c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:54.344184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f6897-718f-4667-9661-e780f3087c12 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:54.357293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 226/470] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:24:54 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:24:54.358621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70268f-ee14-455d-bef3-33f052f83b7f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-596f6897-718f-4667-9661-e780f3087c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:24:54.360642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:54.361581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:24:54.362407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:24:54.362407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:54.362407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:54.362407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 245/471] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:54.373218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb08ab4-67e3-4142-b478-1cc29544c98c None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:54.373678 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb08ab4-67e3-4142-b478-1cc29544c98c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:54.374053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb08ab4-67e3-4142-b478-1cc29544c98c None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:54.375855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 246/472] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:54.376623 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb08ab4-67e3-4142-b478-1cc29544c98c None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:54.412250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:54.507103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d70268f-ee14-455d-bef3-33f052f83b7f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:54.528532 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb08ab4-67e3-4142-b478-1cc29544c98c None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:54.570619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca1c47-e8fb-42e8-9dd3-dcfb090c026a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:54.571786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca1c47-e8fb-42e8-9dd3-dcfb090c026a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:54.572294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca1c47-e8fb-42e8-9dd3-dcfb090c026a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:54.572708 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca1c47-e8fb-42e8-9dd3-dcfb090c026a None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:54.573627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d49578-cd69-4a92-b5a8-110c03cbbecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:54.573782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca1c47-e8fb-42e8-9dd3-dcfb090c026a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:54.574215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca1c47-e8fb-42e8-9dd3-dcfb090c026a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:54.574507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d49578-cd69-4a92-b5a8-110c03cbbecb None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:54.575685 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d49578-cd69-4a92-b5a8-110c03cbbecb None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:54.575685 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d49578-cd69-4a92-b5a8-110c03cbbecb None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:54.575928 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d49578-cd69-4a92-b5a8-110c03cbbecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:54.576388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d49578-cd69-4a92-b5a8-110c03cbbecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:54.577411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca1c47-e8fb-42e8-9dd3-dcfb090c026a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:54.578004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d49578-cd69-4a92-b5a8-110c03cbbecb None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:54.593422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 247/473] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:54.595284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec704ab-7c65-4e07-9261-1cfa1d900575 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:54.595625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec704ab-7c65-4e07-9261-1cfa1d900575 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:54.595905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec704ab-7c65-4e07-9261-1cfa1d900575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:54.604525 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 227/474] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:54.605472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca1c47-e8fb-42e8-9dd3-dcfb090c026a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:54.612636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:54.622014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-6ec704ab-7c65-4e07-9261-1cfa1d900575 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:54.623641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-6ec704ab-7c65-4e07-9261-1cfa1d900575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:54.700805 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 248/475] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:54.702069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec704ab-7c65-4e07-9261-1cfa1d900575 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:54.724970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:54.756988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca1c47-e8fb-42e8-9dd3-dcfb090c026a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:54.825797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edb6bb-3a06-4eb3-8f73-24b9260af65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:54.826328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edb6bb-3a06-4eb3-8f73-24b9260af65f None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:54.826603 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edb6bb-3a06-4eb3-8f73-24b9260af65f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:54.826870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edb6bb-3a06-4eb3-8f73-24b9260af65f None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:54.827519 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edb6bb-3a06-4eb3-8f73-24b9260af65f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:54.829539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edb6bb-3a06-4eb3-8f73-24b9260af65f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:54.868336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 228/476] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:24:54 2022] POST /identity/v3/users => generated 249 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:24:54.871112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec704ab-7c65-4e07-9261-1cfa1d900575 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:54.924888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7426058-3bf4-4049-84dc-48086bcb3b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:54.925566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7426058-3bf4-4049-84dc-48086bcb3b9c None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:54.925849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7426058-3bf4-4049-84dc-48086bcb3b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:54.926121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7426058-3bf4-4049-84dc-48086bcb3b9c None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:54.926791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7426058-3bf4-4049-84dc-48086bcb3b9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:54.928242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7426058-3bf4-4049-84dc-48086bcb3b9c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:54.959876 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 249/477] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:24:54 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:24:55.040056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edb6bb-3a06-4eb3-8f73-24b9260af65f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:55.120433 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:55.171757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edb6bb-3a06-4eb3-8f73-24b9260af65f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:55.212112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ad6c0-fb7d-46a6-b4bd-45a99b2f4216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:55.212731 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ad6c0-fb7d-46a6-b4bd-45a99b2f4216 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:55.213016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ad6c0-fb7d-46a6-b4bd-45a99b2f4216 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:55.213314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ad6c0-fb7d-46a6-b4bd-45a99b2f4216 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:55.214006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ad6c0-fb7d-46a6-b4bd-45a99b2f4216 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:55.215391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ad6c0-fb7d-46a6-b4bd-45a99b2f4216 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:55.225515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.sql.core [None req-011ad6c0-fb7d-46a6-b4bd-45a99b2f4216 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 08:24:55.225515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 28 08:24:55.225515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [parameters: {'id': '9dd596b773814dc598d77a680e61666b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 28 08:24:55.225515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81483) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-011ad6c0-fb7d-46a6-b4bd-45a99b2f4216 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application conn.query(q) Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application result.read() Sep 28 08:24:55.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return ref.to_dict() Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 08:24:55.244199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application next(self.gen) Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application yield resource Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application next(self.gen) Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise self.value Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application session.commit() Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 08:24:55.245127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self.session.flush() Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application compat.raise_( Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 08:24:55.245967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 08:24:55.246811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application conn.query(q) Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application result.read() Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 08:24:55.247620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [parameters: {'id': '9dd596b773814dc598d77a680e61666b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:55.248382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 28 08:24:55.249386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:55.250700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 229/478] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 08:24:55 2022] POST /identity/v3/roles => generated 144 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:55.251714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7426058-3bf4-4049-84dc-48086bcb3b9c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:55.342693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:55.384464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7426058-3bf4-4049-84dc-48086bcb3b9c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:55.422705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbd3d6-9ee4-42d5-a653-61f4a222da43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:55.423860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbd3d6-9ee4-42d5-a653-61f4a222da43 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:55.424124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbd3d6-9ee4-42d5-a653-61f4a222da43 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:55.424395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbd3d6-9ee4-42d5-a653-61f4a222da43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:55.435673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbd3d6-9ee4-42d5-a653-61f4a222da43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-05bbd3d6-9ee4-42d5-a653-61f4a222da43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:55.441069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:55.441979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:55.442767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:55.442767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:55.442767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:55.442767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:55.442767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:55.442767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:55.442767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:55.442767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:55.442767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:55.442767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:55.442767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 08:24:55.442767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:55.447129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 250/479] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:24:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:55.448493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ad6c0-fb7d-46a6-b4bd-45a99b2f4216 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:55.556341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ad6c0-fb7d-46a6-b4bd-45a99b2f4216 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:55.580849 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:55.592305 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6f04b-a800-4767-b7b8-917bef952c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:55.592923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6f04b-a800-4767-b7b8-917bef952c3b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:55.593231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6f04b-a800-4767-b7b8-917bef952c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:55.593502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6f04b-a800-4767-b7b8-917bef952c3b None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:55.594176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6f04b-a800-4767-b7b8-917bef952c3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:55.595062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6f04b-a800-4767-b7b8-917bef952c3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:55.596402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6f04b-a800-4767-b7b8-917bef952c3b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:55.609223 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 230/480] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:56.435142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3396210-4b45-4c62-9fb7-c8cf96c67678 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:56.435662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3396210-4b45-4c62-9fb7-c8cf96c67678 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:56.435770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3396210-4b45-4c62-9fb7-c8cf96c67678 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:56.437353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 251/481] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:56.443182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea72eb-e51b-424b-b207-8c4c43420d69 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:56.443614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea72eb-e51b-424b-b207-8c4c43420d69 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:56.443790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea72eb-e51b-424b-b207-8c4c43420d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:56.463109 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-d9ea72eb-e51b-424b-b207-8c4c43420d69 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:56.464316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d9ea72eb-e51b-424b-b207-8c4c43420d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:56.565210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 231/482] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:56.897652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3396210-4b45-4c62-9fb7-c8cf96c67678 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:56.992677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00f0f4-6ede-444c-91c3-2d8427c1eb9e None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:56.992999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00f0f4-6ede-444c-91c3-2d8427c1eb9e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:56.993255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00f0f4-6ede-444c-91c3-2d8427c1eb9e None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:56.995101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 232/483] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:56.999569 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab757f-61f3-4359-884b-f17105785274 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:56.999875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab757f-61f3-4359-884b-f17105785274 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:57.000121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab757f-61f3-4359-884b-f17105785274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:57.030457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3396210-4b45-4c62-9fb7-c8cf96c67678 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:57.031392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-aeab757f-61f3-4359-884b-f17105785274 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:57.032616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-aeab757f-61f3-4359-884b-f17105785274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:57.077440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:57.088278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9ad81-51ab-4da5-8dd2-b18efdcd1a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:57.088913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9ad81-51ab-4da5-8dd2-b18efdcd1a21 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:57.089211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9ad81-51ab-4da5-8dd2-b18efdcd1a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:57.089480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9ad81-51ab-4da5-8dd2-b18efdcd1a21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:57.110843 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c9ad81-51ab-4da5-8dd2-b18efdcd1a21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-62c9ad81-51ab-4da5-8dd2-b18efdcd1a21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:57.115630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:57.116514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:57.117330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:57.117330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:57.117330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:57.117330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:57.117330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:57.117330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:57.117330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:57.117330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:57.117330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:57.117330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:57.117330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:24:57.117330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:57.118396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 252/484] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:24:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:57.127343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9ad81-51ab-4da5-8dd2-b18efdcd1a21 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:57.147205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 233/485] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:24:57.252934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9ad81-51ab-4da5-8dd2-b18efdcd1a21 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:57.292586 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5866c-7325-49e6-9284-157195c5d580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:57.293222 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5866c-7325-49e6-9284-157195c5d580 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:57.293502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5866c-7325-49e6-9284-157195c5d580 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:57.293777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5866c-7325-49e6-9284-157195c5d580 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:57.294445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5866c-7325-49e6-9284-157195c5d580 None admin] RBAC: Adding query filter params (name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:57.294746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5866c-7325-49e6-9284-157195c5d580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:57.296113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5866c-7325-49e6-9284-157195c5d580 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:57.305206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 253/486] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:24:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:57.314792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab757f-61f3-4359-884b-f17105785274 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:57.392278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:57.465609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab757f-61f3-4359-884b-f17105785274 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:57.508582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5866c-7325-49e6-9284-157195c5d580 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:57.511784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32a93a-1122-4d25-8dc6-848bcaa8aa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:57.512426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32a93a-1122-4d25-8dc6-848bcaa8aa88 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:57.512709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32a93a-1122-4d25-8dc6-848bcaa8aa88 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:57.512981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32a93a-1122-4d25-8dc6-848bcaa8aa88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:57.534150 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32a93a-1122-4d25-8dc6-848bcaa8aa88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:57.535669 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32a93a-1122-4d25-8dc6-848bcaa8aa88 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-8e32a93a-1122-4d25-8dc6-848bcaa8aa88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:24:57.554936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:57.555904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:24:57.557536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:24:57.557536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:57.557536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:57.570147 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 234/487] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:57.574711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32a93a-1122-4d25-8dc6-848bcaa8aa88 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:57.619445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:57.707159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5866c-7325-49e6-9284-157195c5d580 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:57.764284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:57.769028 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32a93a-1122-4d25-8dc6-848bcaa8aa88 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:57.794420 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647fb4d-23d3-4a98-b9ae-9e2af2dce0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:57.795249 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647fb4d-23d3-4a98-b9ae-9e2af2dce0ae None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:57.795877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647fb4d-23d3-4a98-b9ae-9e2af2dce0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:57.796158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647fb4d-23d3-4a98-b9ae-9e2af2dce0ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:57.807108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05698af5-c45d-40c5-a5f9-1892c4ed26f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:57.807723 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05698af5-c45d-40c5-a5f9-1892c4ed26f8 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:57.808004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05698af5-c45d-40c5-a5f9-1892c4ed26f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:57.808279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05698af5-c45d-40c5-a5f9-1892c4ed26f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:57.808928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05698af5-c45d-40c5-a5f9-1892c4ed26f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:57.809326 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05698af5-c45d-40c5-a5f9-1892c4ed26f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:57.810695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05698af5-c45d-40c5-a5f9-1892c4ed26f8 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:57.823730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 235/488] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:57.825307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647fb4d-23d3-4a98-b9ae-9e2af2dce0ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:57.829659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05698af5-c45d-40c5-a5f9-1892c4ed26f8 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-4647fb4d-23d3-4a98-b9ae-9e2af2dce0ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:57.832170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:57.833029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:57.833853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:57.833853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:57.833853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:57.833853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:57.833853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:57.833853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:57.833853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:57.833853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:57.833853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:57.833853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:57.833853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:24:57.833853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:57.837123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 254/489] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:24:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:57.845526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647fb4d-23d3-4a98-b9ae-9e2af2dce0ae None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:57.972576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05698af5-c45d-40c5-a5f9-1892c4ed26f8 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:58.019135 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647fb4d-23d3-4a98-b9ae-9e2af2dce0ae None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:58.022915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7361227-5775-4069-aebe-8d71bbd774b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:58.023562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7361227-5775-4069-aebe-8d71bbd774b5 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.023846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7361227-5775-4069-aebe-8d71bbd774b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.024120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7361227-5775-4069-aebe-8d71bbd774b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.034717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7361227-5775-4069-aebe-8d71bbd774b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:58.036038 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:58.036364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7361227-5775-4069-aebe-8d71bbd774b5 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-f7361227-5775-4069-aebe-8d71bbd774b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:24:58.053466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:58.054573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:58.055504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:58.055504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:58.055504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:24:58.055504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:24:58.055504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:24:58.055504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:24:58.055504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:24:58.055504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:24:58.055504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:24:58.055504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:58.055504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 236/490] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 08:24:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:24:58.056413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68915f13-6623-4dd0-a867-eb14ff0c9b2c None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.056699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68915f13-6623-4dd0-a867-eb14ff0c9b2c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.056982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68915f13-6623-4dd0-a867-eb14ff0c9b2c None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.061161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 237/491] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:58.064792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68915f13-6623-4dd0-a867-eb14ff0c9b2c None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:58.069819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba282ee-308d-482b-99a8-3d46364bae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:58.070472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba282ee-308d-482b-99a8-3d46364bae63 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.070759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba282ee-308d-482b-99a8-3d46364bae63 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.071033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba282ee-308d-482b-99a8-3d46364bae63 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.071708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba282ee-308d-482b-99a8-3d46364bae63 None admin] RBAC: Adding query filter params (name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:58.072013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba282ee-308d-482b-99a8-3d46364bae63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:58.073404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba282ee-308d-482b-99a8-3d46364bae63 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:58.083152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 255/492] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:24:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:24:58.084358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf50f5-1c85-40bc-b46c-274f8bae1f1b None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.084646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf50f5-1c85-40bc-b46c-274f8bae1f1b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.084920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf50f5-1c85-40bc-b46c-274f8bae1f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.091365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:58.110535 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-4fdf50f5-1c85-40bc-b46c-274f8bae1f1b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:58.111715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4fdf50f5-1c85-40bc-b46c-274f8bae1f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:58.186979 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68915f13-6623-4dd0-a867-eb14ff0c9b2c None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:58.201116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 256/493] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:58.201116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf50f5-1c85-40bc-b46c-274f8bae1f1b None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:58.237113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5e499-9d12-4d54-a4ab-3c4a29fe96c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:58.237842 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5e499-9d12-4d54-a4ab-3c4a29fe96c0 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.238194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5e499-9d12-4d54-a4ab-3c4a29fe96c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.238540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5e499-9d12-4d54-a4ab-3c4a29fe96c0 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.239352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5e499-9d12-4d54-a4ab-3c4a29fe96c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:58.239729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5e499-9d12-4d54-a4ab-3c4a29fe96c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:58.241371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5e499-9d12-4d54-a4ab-3c4a29fe96c0 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:58.282807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 238/494] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 08:24:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:58.284145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edcd9f-4956-45c2-aad3-458c4ba61921 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.284509 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edcd9f-4956-45c2-aad3-458c4ba61921 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.284853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edcd9f-4956-45c2-aad3-458c4ba61921 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.286415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 239/495] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:58.291255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edcd9f-4956-45c2-aad3-458c4ba61921 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:58.325060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf50f5-1c85-40bc-b46c-274f8bae1f1b None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:58.335738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:58.376049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f937b-0f24-4d45-9dbc-9e4af9f86440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:58.377275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f937b-0f24-4d45-9dbc-9e4af9f86440 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.377709 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f937b-0f24-4d45-9dbc-9e4af9f86440 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.381476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f937b-0f24-4d45-9dbc-9e4af9f86440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.384877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:58.398061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f937b-0f24-4d45-9dbc-9e4af9f86440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:58.400484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f937b-0f24-4d45-9dbc-9e4af9f86440 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:58.413255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edcd9f-4956-45c2-aad3-458c4ba61921 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-3d6f937b-0f24-4d45-9dbc-9e4af9f86440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:24:58.422197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:58.423722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:24:58.424544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:24:58.424544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:58.424544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:58.424544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 257/496] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:58.426138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718d63d-228f-4cf2-ade1-4dfb872cd1e8 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.426497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718d63d-228f-4cf2-ade1-4dfb872cd1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.426844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718d63d-228f-4cf2-ade1-4dfb872cd1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.453689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-6718d63d-228f-4cf2-ade1-4dfb872cd1e8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:58.455706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-6718d63d-228f-4cf2-ade1-4dfb872cd1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:58.458757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadd2b3-90b9-40b4-b97b-3a409f14b938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:58.459703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadd2b3-90b9-40b4-b97b-3a409f14b938 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.460187 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadd2b3-90b9-40b4-b97b-3a409f14b938 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.461059 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadd2b3-90b9-40b4-b97b-3a409f14b938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.538980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadd2b3-90b9-40b4-b97b-3a409f14b938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:58.540372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadd2b3-90b9-40b4-b97b-3a409f14b938 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:58.549610 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 258/497] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:58.551615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718d63d-228f-4cf2-ade1-4dfb872cd1e8 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:58.631232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:58.656428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:58.663244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 240/498] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:58.663244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadd2b3-90b9-40b4-b97b-3a409f14b938 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:58.675644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718d63d-228f-4cf2-ade1-4dfb872cd1e8 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:58.715267 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088b793-86b9-49e4-b9c4-cce37e16b6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:58.715885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088b793-86b9-49e4-b9c4-cce37e16b6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.716171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088b793-86b9-49e4-b9c4-cce37e16b6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.716444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088b793-86b9-49e4-b9c4-cce37e16b6d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.717117 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088b793-86b9-49e4-b9c4-cce37e16b6d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:58.717462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088b793-86b9-49e4-b9c4-cce37e16b6d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:58.718842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088b793-86b9-49e4-b9c4-cce37e16b6d6 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:58.732055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 259/499] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:58.737594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ea1f8-f89a-405e-a41e-0a0a8ad15187 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.737886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ea1f8-f89a-405e-a41e-0a0a8ad15187 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.738167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ea1f8-f89a-405e-a41e-0a0a8ad15187 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.739416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 260/500] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:24:58.740120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ea1f8-f89a-405e-a41e-0a0a8ad15187 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:58.770341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadd2b3-90b9-40b4-b97b-3a409f14b938 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:58.806790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360ca54-11d1-4314-9a80-086e24bf2ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:58.807689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360ca54-11d1-4314-9a80-086e24bf2ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.808160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360ca54-11d1-4314-9a80-086e24bf2ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.808680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360ca54-11d1-4314-9a80-086e24bf2ca4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.819641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360ca54-11d1-4314-9a80-086e24bf2ca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-4360ca54-11d1-4314-9a80-086e24bf2ca4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:58.824491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:58.825602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 241/501] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:24:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:58.826407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360ca54-11d1-4314-9a80-086e24bf2ca4 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:58.848177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ea1f8-f89a-405e-a41e-0a0a8ad15187 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:58.862721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:58.882133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2909890-be3d-4220-b4e8-ea4fedef6223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:58.882762 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2909890-be3d-4220-b4e8-ea4fedef6223 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.883042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2909890-be3d-4220-b4e8-ea4fedef6223 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.883314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2909890-be3d-4220-b4e8-ea4fedef6223 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.894559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2909890-be3d-4220-b4e8-ea4fedef6223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:58.896004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2909890-be3d-4220-b4e8-ea4fedef6223 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:58.904361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-a2909890-be3d-4220-b4e8-ea4fedef6223 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:24:58.912582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:58.913550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:24:58.914369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:24:58.914369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:58.914369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:58.914369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 261/502] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:58.915167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82360dbe-cbdf-4bb4-bd39-64ddf301c892 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.915167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82360dbe-cbdf-4bb4-bd39-64ddf301c892 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.915391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82360dbe-cbdf-4bb4-bd39-64ddf301c892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.930537 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360ca54-11d1-4314-9a80-086e24bf2ca4 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:58.934316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-82360dbe-cbdf-4bb4-bd39-64ddf301c892 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:24:58.935511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-82360dbe-cbdf-4bb4-bd39-64ddf301c892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:24:58.965682 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197094c-2f9b-4e2e-9d4b-44c8e8611e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:58.966306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197094c-2f9b-4e2e-9d4b-44c8e8611e6b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:58.966586 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197094c-2f9b-4e2e-9d4b-44c8e8611e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:58.966856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197094c-2f9b-4e2e-9d4b-44c8e8611e6b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:58.973986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197094c-2f9b-4e2e-9d4b-44c8e8611e6b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:58.975424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197094c-2f9b-4e2e-9d4b-44c8e8611e6b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-f197094c-2f9b-4e2e-9d4b-44c8e8611e6b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:24:58.987202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:58.988190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:58.994023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:58.994023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:58.994023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:24:58.994023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:24:58.994023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:24:58.994023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:24:58.994023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:24:58.994023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:24:58.994023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 08:24:58.994023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:58.994023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 242/503] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:24:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:58.994023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197094c-2f9b-4e2e-9d4b-44c8e8611e6b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.002683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 262/504] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:24:59.003523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82360dbe-cbdf-4bb4-bd39-64ddf301c892 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.104930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:59.111527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197094c-2f9b-4e2e-9d4b-44c8e8611e6b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:59.111726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82360dbe-cbdf-4bb4-bd39-64ddf301c892 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:59.143431 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:59.149681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d906e-f4bf-4e47-b6a6-8971c3846858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:59.150000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f8317-c611-400a-a8e9-c54c27c4eab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:59.150611 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f8317-c611-400a-a8e9-c54c27c4eab5 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:59.150841 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d906e-f4bf-4e47-b6a6-8971c3846858 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:59.151200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f8317-c611-400a-a8e9-c54c27c4eab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:59.151399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f8317-c611-400a-a8e9-c54c27c4eab5 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:59.151758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d906e-f4bf-4e47-b6a6-8971c3846858 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:59.152059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f8317-c611-400a-a8e9-c54c27c4eab5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:59.152328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d906e-f4bf-4e47-b6a6-8971c3846858 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:59.152437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f8317-c611-400a-a8e9-c54c27c4eab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:59.153734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f8317-c611-400a-a8e9-c54c27c4eab5 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:59.161803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 263/505] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:24:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:59.162594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f8317-c611-400a-a8e9-c54c27c4eab5 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.167368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d906e-f4bf-4e47-b6a6-8971c3846858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-280d906e-f4bf-4e47-b6a6-8971c3846858 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:24:59.172703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:59.173604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:59.174372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 243/506] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:24:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:59.175031 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d906e-f4bf-4e47-b6a6-8971c3846858 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.277679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d906e-f4bf-4e47-b6a6-8971c3846858 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:59.277874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f8317-c611-400a-a8e9-c54c27c4eab5 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:59.309921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43289cbc-2bda-4c69-b795-867090b7b2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:59.310425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1fade-a492-4c51-b963-2010ec4edc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:59.310565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43289cbc-2bda-4c69-b795-867090b7b2cd None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:59.310779 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43289cbc-2bda-4c69-b795-867090b7b2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:59.311024 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1fade-a492-4c51-b963-2010ec4edc2c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:59.311113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43289cbc-2bda-4c69-b795-867090b7b2cd None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:59.311304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1fade-a492-4c51-b963-2010ec4edc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:59.311580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1fade-a492-4c51-b963-2010ec4edc2c None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:59.311777 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43289cbc-2bda-4c69-b795-867090b7b2cd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:59.312079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43289cbc-2bda-4c69-b795-867090b7b2cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:59.312212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1fade-a492-4c51-b963-2010ec4edc2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:59.312517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1fade-a492-4c51-b963-2010ec4edc2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:59.314057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43289cbc-2bda-4c69-b795-867090b7b2cd None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:59.314397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1fade-a492-4c51-b963-2010ec4edc2c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:59.326246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 264/507] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:59.331313 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1fade-a492-4c51-b963-2010ec4edc2c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.335090 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 244/508] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 08:24:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:59.336357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43289cbc-2bda-4c69-b795-867090b7b2cd None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.352892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:59.381738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:59.429591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1fade-a492-4c51-b963-2010ec4edc2c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:59.439446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43289cbc-2bda-4c69-b795-867090b7b2cd None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:59.465745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d908e-2211-4366-9d96-30b52ce6fac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:59.466376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d908e-2211-4366-9d96-30b52ce6fac4 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:59.466665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d908e-2211-4366-9d96-30b52ce6fac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:59.466935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d908e-2211-4366-9d96-30b52ce6fac4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:59.473572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a071348-ac40-4565-b8a2-7285d7210b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:59.474174 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a071348-ac40-4565-b8a2-7285d7210b5b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:59.474454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a071348-ac40-4565-b8a2-7285d7210b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:59.474727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a071348-ac40-4565-b8a2-7285d7210b5b None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:59.475391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a071348-ac40-4565-b8a2-7285d7210b5b None admin] RBAC: Adding query filter params (name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:59.475694 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a071348-ac40-4565-b8a2-7285d7210b5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:59.477049 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a071348-ac40-4565-b8a2-7285d7210b5b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:59.477492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d908e-2211-4366-9d96-30b52ce6fac4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:59.478921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d908e-2211-4366-9d96-30b52ce6fac4 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:59.485750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 245/509] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:24:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:59.486551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a071348-ac40-4565-b8a2-7285d7210b5b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-971d908e-2211-4366-9d96-30b52ce6fac4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:24:59.494700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:59.495775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:24:59.496782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:24:59.496782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:59.496782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:59.496782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 265/510] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:59.496782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d908e-2211-4366-9d96-30b52ce6fac4 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.570928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:59.597417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a071348-ac40-4565-b8a2-7285d7210b5b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:59.607353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d908e-2211-4366-9d96-30b52ce6fac4 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:59.610871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:59.633041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5619f3b-9d73-47a3-935d-33acbdecd62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:59.633730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5619f3b-9d73-47a3-935d-33acbdecd62a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:59.634033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5619f3b-9d73-47a3-935d-33acbdecd62a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:59.634362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5619f3b-9d73-47a3-935d-33acbdecd62a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:59.644437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edccbd-cfab-424d-b51e-46417e9178e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:59.645075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edccbd-cfab-424d-b51e-46417e9178e6 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:59.645331 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5619f3b-9d73-47a3-935d-33acbdecd62a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:59.645457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edccbd-cfab-424d-b51e-46417e9178e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:59.645649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edccbd-cfab-424d-b51e-46417e9178e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:59.646773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5619f3b-9d73-47a3-935d-33acbdecd62a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:59.652651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edccbd-cfab-424d-b51e-46417e9178e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:59.654107 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edccbd-cfab-424d-b51e-46417e9178e6 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-e5619f3b-9d73-47a3-935d-33acbdecd62a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:24:59.661188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:59.662056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:24:59.662907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:24:59.662907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:59.662907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:24:59.662907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 246/511] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:59.663117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5619f3b-9d73-47a3-935d-33acbdecd62a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-88edccbd-cfab-424d-b51e-46417e9178e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:24:59.666680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:59.667696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:59.668530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:24:59.668530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:24:59.668530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:24:59.668530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:24:59.668530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:24:59.668530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:24:59.668530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:24:59.668530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:59.668530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 266/512] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:24:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:59.669409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edccbd-cfab-424d-b51e-46417e9178e6 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.763303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5619f3b-9d73-47a3-935d-33acbdecd62a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:59.766671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edccbd-cfab-424d-b51e-46417e9178e6 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:59.789639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:59.798164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca289af-f770-4e8f-909b-39c379c7538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:59.798768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca289af-f770-4e8f-909b-39c379c7538f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:59.799043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca289af-f770-4e8f-909b-39c379c7538f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:59.799311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca289af-f770-4e8f-909b-39c379c7538f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:59.801384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e6108-b201-47ca-897d-b90299210e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:59.802006 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e6108-b201-47ca-897d-b90299210e74 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:59.802284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e6108-b201-47ca-897d-b90299210e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:59.802559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e6108-b201-47ca-897d-b90299210e74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:59.813203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e6108-b201-47ca-897d-b90299210e74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:59.814654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e6108-b201-47ca-897d-b90299210e74 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-a51e6108-b201-47ca-897d-b90299210e74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:24:59.828250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:24:59.829176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:24:59.830015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:24:59.830015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:24:59.830015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:24:59.830015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 267/513] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:24:59.830236 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e6108-b201-47ca-897d-b90299210e74 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.851738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:24:59.868501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca289af-f770-4e8f-909b-39c379c7538f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:59.869988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca289af-f770-4e8f-909b-39c379c7538f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:59.933524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e6108-b201-47ca-897d-b90299210e74 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:24:59.966121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def59a0-d651-492f-8be8-5000cb074e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:24:59.966722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def59a0-d651-492f-8be8-5000cb074e18 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:24:59.966844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 247/514] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:24:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:24:59.967028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def59a0-d651-492f-8be8-5000cb074e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:24:59.967270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def59a0-d651-492f-8be8-5000cb074e18 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:24:59.967607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca289af-f770-4e8f-909b-39c379c7538f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.967913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def59a0-d651-492f-8be8-5000cb074e18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:24:59.968219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def59a0-d651-492f-8be8-5000cb074e18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:24:59.970157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def59a0-d651-492f-8be8-5000cb074e18 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:24:59.982932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 268/515] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:24:59.983742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def59a0-d651-492f-8be8-5000cb074e18 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:24:59.995371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:00.061281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:00.067490 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca289af-f770-4e8f-909b-39c379c7538f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:00.088385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def59a0-d651-492f-8be8-5000cb074e18 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:00.102666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c22a7-36cb-487c-8d1e-ed6c7a70b796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:00.103269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c22a7-36cb-487c-8d1e-ed6c7a70b796 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:00.103544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c22a7-36cb-487c-8d1e-ed6c7a70b796 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:00.103813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c22a7-36cb-487c-8d1e-ed6c7a70b796 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:00.104446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c22a7-36cb-487c-8d1e-ed6c7a70b796 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:00.104743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c22a7-36cb-487c-8d1e-ed6c7a70b796 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:00.106122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c22a7-36cb-487c-8d1e-ed6c7a70b796 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:00.117665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 248/516] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:00.118469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c22a7-36cb-487c-8d1e-ed6c7a70b796 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:00.127233 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0dcae-37c4-4e5a-8e22-1f42612e2040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:00.127845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0dcae-37c4-4e5a-8e22-1f42612e2040 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:00.128126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0dcae-37c4-4e5a-8e22-1f42612e2040 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:00.128402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0dcae-37c4-4e5a-8e22-1f42612e2040 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:00.129109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0dcae-37c4-4e5a-8e22-1f42612e2040 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:00.129428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0dcae-37c4-4e5a-8e22-1f42612e2040 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:00.130853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0dcae-37c4-4e5a-8e22-1f42612e2040 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:00.149291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 269/517] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:24:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:00.150098 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0dcae-37c4-4e5a-8e22-1f42612e2040 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:00.209593 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:00.222795 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c22a7-36cb-487c-8d1e-ed6c7a70b796 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:00.249590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0dcae-37c4-4e5a-8e22-1f42612e2040 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:00.255379 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ae096-f7b4-47d6-a534-abf023a5c3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:00.255988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ae096-f7b4-47d6-a534-abf023a5c3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:00.256268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ae096-f7b4-47d6-a534-abf023a5c3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:00.256539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ae096-f7b4-47d6-a534-abf023a5c3b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:00.257207 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ae096-f7b4-47d6-a534-abf023a5c3b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:00.257509 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ae096-f7b4-47d6-a534-abf023a5c3b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:00.258877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ae096-f7b4-47d6-a534-abf023a5c3b4 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:00.270999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 249/518] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:00.271798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ae096-f7b4-47d6-a534-abf023a5c3b4 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:00.284643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac00af-9671-4d74-af4a-0b6f129365f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:00.285306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac00af-9671-4d74-af4a-0b6f129365f7 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:00.285593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac00af-9671-4d74-af4a-0b6f129365f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:00.285863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac00af-9671-4d74-af4a-0b6f129365f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:00.288613 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:00.297330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac00af-9671-4d74-af4a-0b6f129365f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:00.298835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac00af-9671-4d74-af4a-0b6f129365f7 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-53ac00af-9671-4d74-af4a-0b6f129365f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:00.314430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:00.315339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:00.316331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:00.316331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:00.316331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:00.316331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 270/519] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:00.316331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac00af-9671-4d74-af4a-0b6f129365f7 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:00.380112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ae096-f7b4-47d6-a534-abf023a5c3b4 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:00.414319 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e2be8-e8d6-478c-a28c-a0c8309afa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:00.414945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e2be8-e8d6-478c-a28c-a0c8309afa46 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:00.415228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e2be8-e8d6-478c-a28c-a0c8309afa46 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:00.415503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e2be8-e8d6-478c-a28c-a0c8309afa46 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:00.419369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac00af-9671-4d74-af4a-0b6f129365f7 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:00.422686 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e2be8-e8d6-478c-a28c-a0c8309afa46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:00.424146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e2be8-e8d6-478c-a28c-a0c8309afa46 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-fa6e2be8-e8d6-478c-a28c-a0c8309afa46 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:00.435791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:00.436800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:00.437735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:00.437735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:00.437735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:00.437735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:00.437735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:00.437735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:00.437735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:00.437735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:00.437735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 08:25:00.437735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:00.437735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 250/520] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:25:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:00.437735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e2be8-e8d6-478c-a28c-a0c8309afa46 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:00.448914 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:00.460197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18e468-a3ce-49e1-b274-e20cd8d67795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:00.460800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18e468-a3ce-49e1-b274-e20cd8d67795 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:00.461080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18e468-a3ce-49e1-b274-e20cd8d67795 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:00.461807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18e468-a3ce-49e1-b274-e20cd8d67795 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:00.462459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18e468-a3ce-49e1-b274-e20cd8d67795 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:00.463875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18e468-a3ce-49e1-b274-e20cd8d67795 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:00.484465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 271/521] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:25:00 2022] POST /identity/v3/users => generated 247 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:25:00.486235 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18e468-a3ce-49e1-b274-e20cd8d67795 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:00.551641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:00.555431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e2be8-e8d6-478c-a28c-a0c8309afa46 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:00.600167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d0990-04c4-4837-bbb4-6edebf2c5a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:00.600842 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d0990-04c4-4837-bbb4-6edebf2c5a84 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:00.601131 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d0990-04c4-4837-bbb4-6edebf2c5a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:00.601336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d0990-04c4-4837-bbb4-6edebf2c5a84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:00.602221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d0990-04c4-4837-bbb4-6edebf2c5a84 None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=9c3c6efc3c154ff89066035cd9583459) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:00.602505 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d0990-04c4-4837-bbb4-6edebf2c5a84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:00.603973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d0990-04c4-4837-bbb4-6edebf2c5a84 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:00.620987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 251/522] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:00 2022] GET /identity/v3/role_assignments?user.id=9c3c6efc3c154ff89066035cd9583459&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:00.620987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d0990-04c4-4837-bbb4-6edebf2c5a84 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:00.625012 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18e468-a3ce-49e1-b274-e20cd8d67795 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:00.668663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9294ef-7977-427f-b46c-de3cbb1aa977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:00.669606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9294ef-7977-427f-b46c-de3cbb1aa977 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:00.669881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9294ef-7977-427f-b46c-de3cbb1aa977 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:00.670151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9294ef-7977-427f-b46c-de3cbb1aa977 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:00.679370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9294ef-7977-427f-b46c-de3cbb1aa977 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:00.680817 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9294ef-7977-427f-b46c-de3cbb1aa977 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-2b9294ef-7977-427f-b46c-de3cbb1aa977 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:00.699073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:00.700005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:00.700891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:00.700891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:00.700891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:00.700891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:00.700891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:00.700891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:00.700891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:00.700891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:00.700891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 08:25:00.700891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:00.700891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 272/523] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:00.701496 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9294ef-7977-427f-b46c-de3cbb1aa977 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:00.721167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:00.761937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d0990-04c4-4837-bbb4-6edebf2c5a84 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:00.819627 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448c398-32ee-4ef9-946d-1a069c3bcac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:00.820900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448c398-32ee-4ef9-946d-1a069c3bcac0 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:00.821684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448c398-32ee-4ef9-946d-1a069c3bcac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:00.822271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448c398-32ee-4ef9-946d-1a069c3bcac0 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:00.823568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448c398-32ee-4ef9-946d-1a069c3bcac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:00.823879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448c398-32ee-4ef9-946d-1a069c3bcac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:00.825291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448c398-32ee-4ef9-946d-1a069c3bcac0 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:00.841110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 252/524] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:00.842066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448c398-32ee-4ef9-946d-1a069c3bcac0 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:00.843742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9294ef-7977-427f-b46c-de3cbb1aa977 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:00.846951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:00.889906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba789dc3-1b63-4a38-9602-1fb064932c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:00.891061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba789dc3-1b63-4a38-9602-1fb064932c61 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:00.892071 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba789dc3-1b63-4a38-9602-1fb064932c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:00.892523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba789dc3-1b63-4a38-9602-1fb064932c61 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:00.893742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba789dc3-1b63-4a38-9602-1fb064932c61 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:00.894233 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba789dc3-1b63-4a38-9602-1fb064932c61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:00.896699 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba789dc3-1b63-4a38-9602-1fb064932c61 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:00.947354 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 273/525] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 08:25:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:00.948793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba789dc3-1b63-4a38-9602-1fb064932c61 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:00.951743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448c398-32ee-4ef9-946d-1a069c3bcac0 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:00.988762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:00.996530 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d8379-4c56-46c2-a00e-ffcceda23ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:00.997162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d8379-4c56-46c2-a00e-ffcceda23ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:00.997447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d8379-4c56-46c2-a00e-ffcceda23ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:00.997720 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d8379-4c56-46c2-a00e-ffcceda23ef0 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:00.998461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d8379-4c56-46c2-a00e-ffcceda23ef0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:00.998775 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d8379-4c56-46c2-a00e-ffcceda23ef0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:01.001380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d8379-4c56-46c2-a00e-ffcceda23ef0 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:01.027252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 253/526] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Sep 28 08:25:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:01.027458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d8379-4c56-46c2-a00e-ffcceda23ef0 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:01.071423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba789dc3-1b63-4a38-9602-1fb064932c61 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:01.112996 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e96964-0ac4-45a0-bed7-b5b10cfe7759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:01.114307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e96964-0ac4-45a0-bed7-b5b10cfe7759 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:01.114833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e96964-0ac4-45a0-bed7-b5b10cfe7759 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:01.115368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e96964-0ac4-45a0-bed7-b5b10cfe7759 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:01.123321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e96964-0ac4-45a0-bed7-b5b10cfe7759 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:01.125260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e96964-0ac4-45a0-bed7-b5b10cfe7759 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-99e96964-0ac4-45a0-bed7-b5b10cfe7759 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:01.140923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:01.142593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:01.143514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:01.143514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:01.143514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:01.143514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:01.143514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:01.143514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:01.143514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:01.143514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:01.143514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 274/527] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:01.143514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e96964-0ac4-45a0-bed7-b5b10cfe7759 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:01.158239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d8379-4c56-46c2-a00e-ffcceda23ef0 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:01.170059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:01.206429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06473f-024c-4a8e-be7f-76cfb75c1597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:01.207040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06473f-024c-4a8e-be7f-76cfb75c1597 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:01.207322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06473f-024c-4a8e-be7f-76cfb75c1597 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:01.207593 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06473f-024c-4a8e-be7f-76cfb75c1597 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:01.265922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e96964-0ac4-45a0-bed7-b5b10cfe7759 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:01.273067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:01.285745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06473f-024c-4a8e-be7f-76cfb75c1597 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:01.287517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06473f-024c-4a8e-be7f-76cfb75c1597 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:01.311863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5087d3-d6c6-4479-8e47-4c57c3d550bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:01.311863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5087d3-d6c6-4479-8e47-4c57c3d550bf None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:01.311863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5087d3-d6c6-4479-8e47-4c57c3d550bf None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:01.311863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5087d3-d6c6-4479-8e47-4c57c3d550bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:01.396763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5087d3-d6c6-4479-8e47-4c57c3d550bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:01.398299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5087d3-d6c6-4479-8e47-4c57c3d550bf None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:01.428888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 254/528] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:25:01.429507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:01.429860 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06473f-024c-4a8e-be7f-76cfb75c1597 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:01.524906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 275/529] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:25:01.529353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5087d3-d6c6-4479-8e47-4c57c3d550bf None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:01.540910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:01.554921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06473f-024c-4a8e-be7f-76cfb75c1597 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:01.606051 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50a298-ddc8-4e8f-8e5b-9c849aab825d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:01.607368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50a298-ddc8-4e8f-8e5b-9c849aab825d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:01.607785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50a298-ddc8-4e8f-8e5b-9c849aab825d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:01.608208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50a298-ddc8-4e8f-8e5b-9c849aab825d None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:01.609035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50a298-ddc8-4e8f-8e5b-9c849aab825d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:01.609582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50a298-ddc8-4e8f-8e5b-9c849aab825d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:01.614722 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c50a298-ddc8-4e8f-8e5b-9c849aab825d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:01.635913 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 255/530] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:01.637033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50a298-ddc8-4e8f-8e5b-9c849aab825d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:01.646184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5087d3-d6c6-4479-8e47-4c57c3d550bf None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:01.682080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94850f-d9ad-4fe1-8669-e80547c913eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:01.682713 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94850f-d9ad-4fe1-8669-e80547c913eb None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:01.682993 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94850f-d9ad-4fe1-8669-e80547c913eb None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:01.683265 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94850f-d9ad-4fe1-8669-e80547c913eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:01.694908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94850f-d9ad-4fe1-8669-e80547c913eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:01.696381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94850f-d9ad-4fe1-8669-e80547c913eb None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:01.706534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-cc94850f-d9ad-4fe1-8669-e80547c913eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:01.733501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:01.734460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:01.735814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:01.735814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:01.735814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:01.737872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 276/531] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:01.738582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94850f-d9ad-4fe1-8669-e80547c913eb None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:01.800630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c50a298-ddc8-4e8f-8e5b-9c849aab825d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:01.854478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b1695-5c53-4423-920e-72627e55773b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:01.855509 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b1695-5c53-4423-920e-72627e55773b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:01.855984 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b1695-5c53-4423-920e-72627e55773b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:01.856450 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b1695-5c53-4423-920e-72627e55773b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:01.877875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94850f-d9ad-4fe1-8669-e80547c913eb None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:01.886145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b1695-5c53-4423-920e-72627e55773b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:01.887942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b1695-5c53-4423-920e-72627e55773b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-579b1695-5c53-4423-920e-72627e55773b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:01.910740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:01.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:01.912497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:01.912497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:01.912497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:01.919617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 256/532] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:01.919617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b1695-5c53-4423-920e-72627e55773b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:01.923209 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:01.927763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164eeb4-a5cf-4754-a36d-6e9dea6d2220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:01.928652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164eeb4-a5cf-4754-a36d-6e9dea6d2220 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:01.929306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164eeb4-a5cf-4754-a36d-6e9dea6d2220 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:01.929806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164eeb4-a5cf-4754-a36d-6e9dea6d2220 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:01.930751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164eeb4-a5cf-4754-a36d-6e9dea6d2220 None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=5274740e5d7c4ab1ba9e9781815532c9) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:01.931257 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164eeb4-a5cf-4754-a36d-6e9dea6d2220 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:01.932897 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164eeb4-a5cf-4754-a36d-6e9dea6d2220 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:01.943475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 277/533] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:01 2022] GET /identity/v3/role_assignments?user.id=5274740e5d7c4ab1ba9e9781815532c9&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:01.944593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164eeb4-a5cf-4754-a36d-6e9dea6d2220 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:02.021238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:02.148302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b1695-5c53-4423-920e-72627e55773b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:02.167079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164eeb4-a5cf-4754-a36d-6e9dea6d2220 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:02.224687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f0d0b-f7a6-449a-bf51-130cc35a95b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:02.225664 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f0d0b-f7a6-449a-bf51-130cc35a95b6 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:02.226156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f0d0b-f7a6-449a-bf51-130cc35a95b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:02.227410 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f0d0b-f7a6-449a-bf51-130cc35a95b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:02.228268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f0d0b-f7a6-449a-bf51-130cc35a95b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:02.228765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f0d0b-f7a6-449a-bf51-130cc35a95b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:02.230391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f0d0b-f7a6-449a-bf51-130cc35a95b6 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:02.244874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 257/534] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:02.253339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f0d0b-f7a6-449a-bf51-130cc35a95b6 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:02.253480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4c84c-765b-43ba-8b0f-1dbae4f46d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:02.253999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4c84c-765b-43ba-8b0f-1dbae4f46d0f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:02.254286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4c84c-765b-43ba-8b0f-1dbae4f46d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:02.254566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4c84c-765b-43ba-8b0f-1dbae4f46d0f None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:02.257141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4c84c-765b-43ba-8b0f-1dbae4f46d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:02.257141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4c84c-765b-43ba-8b0f-1dbae4f46d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:02.257141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4c84c-765b-43ba-8b0f-1dbae4f46d0f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:02.284513 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 278/535] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:02.293784 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4c84c-765b-43ba-8b0f-1dbae4f46d0f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:02.372661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:02.428821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f0d0b-f7a6-449a-bf51-130cc35a95b6 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:02.476534 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58397856-e872-417a-8396-3611dd5f21fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:02.477492 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58397856-e872-417a-8396-3611dd5f21fd None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:02.477931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58397856-e872-417a-8396-3611dd5f21fd None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:02.478354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58397856-e872-417a-8396-3611dd5f21fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:02.492214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58397856-e872-417a-8396-3611dd5f21fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:02.494149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58397856-e872-417a-8396-3611dd5f21fd None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:02.503735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:02.509676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4c84c-765b-43ba-8b0f-1dbae4f46d0f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-58397856-e872-417a-8396-3611dd5f21fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:02.518301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:02.519198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:02.520043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:02.520043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:02.520043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:02.520043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:02.520043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:02.520043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:02.520043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:02.520043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:02.524628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 258/536] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:02.530760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58397856-e872-417a-8396-3611dd5f21fd None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:02.570466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e66452-8b5d-4197-b071-283114975093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:02.571130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e66452-8b5d-4197-b071-283114975093 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:02.571415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e66452-8b5d-4197-b071-283114975093 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:02.571694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e66452-8b5d-4197-b071-283114975093 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:02.586355 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e66452-8b5d-4197-b071-283114975093 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:02.587830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e66452-8b5d-4197-b071-283114975093 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-78e66452-8b5d-4197-b071-283114975093 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:02.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:02.609524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:02.610412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:02.610412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:02.610412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:02.610412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:02.610412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:02.610412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:02.610412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:02.610412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:02.618185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 279/537] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:02.630123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e66452-8b5d-4197-b071-283114975093 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:02.668635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58397856-e872-417a-8396-3611dd5f21fd None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:02.717145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:02.731405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef362ba8-e047-4f2a-b7a6-15a0bd858f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:02.732366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef362ba8-e047-4f2a-b7a6-15a0bd858f93 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:02.732874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef362ba8-e047-4f2a-b7a6-15a0bd858f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:02.733376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef362ba8-e047-4f2a-b7a6-15a0bd858f93 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:02.734411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef362ba8-e047-4f2a-b7a6-15a0bd858f93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:02.734963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef362ba8-e047-4f2a-b7a6-15a0bd858f93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:02.736686 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef362ba8-e047-4f2a-b7a6-15a0bd858f93 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:02.752819 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 259/538] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:02.760810 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef362ba8-e047-4f2a-b7a6-15a0bd858f93 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:02.863400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e66452-8b5d-4197-b071-283114975093 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:02.915728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef362ba8-e047-4f2a-b7a6-15a0bd858f93 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:02.937207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f964bb1-ec61-4e90-b938-874c668eba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:02.938724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f964bb1-ec61-4e90-b938-874c668eba99 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:02.939463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f964bb1-ec61-4e90-b938-874c668eba99 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:02.939935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f964bb1-ec61-4e90-b938-874c668eba99 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:02.943205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f964bb1-ec61-4e90-b938-874c668eba99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:02.945190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f964bb1-ec61-4e90-b938-874c668eba99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:02.948453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f964bb1-ec61-4e90-b938-874c668eba99 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:02.974718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 280/539] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:02.981072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f964bb1-ec61-4e90-b938-874c668eba99 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:02.988564 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c612cba-d63f-49be-860b-36ce57b674e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:02.994316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c612cba-d63f-49be-860b-36ce57b674e5 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:02.994958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c612cba-d63f-49be-860b-36ce57b674e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:02.995432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c612cba-d63f-49be-860b-36ce57b674e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:02.996422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c612cba-d63f-49be-860b-36ce57b674e5 None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=ff22c2632c614fca8008feab93c8fa2d) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:02.996946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c612cba-d63f-49be-860b-36ce57b674e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:02.998763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c612cba-d63f-49be-860b-36ce57b674e5 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:03.007542 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 260/540] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:02 2022] GET /identity/v3/role_assignments?user.id=ff22c2632c614fca8008feab93c8fa2d&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:03.008909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f846f8-3f7b-4c9c-bb2a-4b7dab3ccf86 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:03.009108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f846f8-3f7b-4c9c-bb2a-4b7dab3ccf86 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:03.009370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f846f8-3f7b-4c9c-bb2a-4b7dab3ccf86 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:03.010565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 261/541] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:03.016290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065437fc-5aee-4fe0-9b1e-11053031da84 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:03.017055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065437fc-5aee-4fe0-9b1e-11053031da84 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:03.017365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065437fc-5aee-4fe0-9b1e-11053031da84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:03.024908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:03.037548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-065437fc-5aee-4fe0-9b1e-11053031da84 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:03.038761 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-065437fc-5aee-4fe0-9b1e-11053031da84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:03.121348 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:03.149864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 262/542] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:25:03.158868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d395f-ec49-42c3-9f9b-89430f4bd8a0 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:03.159172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d395f-ec49-42c3-9f9b-89430f4bd8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:03.159425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d395f-ec49-42c3-9f9b-89430f4bd8a0 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:03.160564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f964bb1-ec61-4e90-b938-874c668eba99 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:03.161102 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 263/543] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:03.166751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cb5c9-bcff-4015-9aa9-9fea02c625ef None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:03.167040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cb5c9-bcff-4015-9aa9-9fea02c625ef None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:03.167412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cb5c9-bcff-4015-9aa9-9fea02c625ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:03.187946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-934cb5c9-bcff-4015-9aa9-9fea02c625ef None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:03.189790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-934cb5c9-bcff-4015-9aa9-9fea02c625ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:03.216495 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08570ffc-b932-4cef-8a67-0ec2f17f5643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:03.216495 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08570ffc-b932-4cef-8a67-0ec2f17f5643 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:03.216915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08570ffc-b932-4cef-8a67-0ec2f17f5643 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:03.217107 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08570ffc-b932-4cef-8a67-0ec2f17f5643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:03.222838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08570ffc-b932-4cef-8a67-0ec2f17f5643 None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=736efafe9fc94b45bc75566b1696c498) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:03.223163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08570ffc-b932-4cef-8a67-0ec2f17f5643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:03.224638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08570ffc-b932-4cef-8a67-0ec2f17f5643 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:03.231397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 281/544] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:02 2022] GET /identity/v3/role_assignments?user.id=736efafe9fc94b45bc75566b1696c498&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:03.283052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 264/545] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:25:03.473771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08570ffc-b932-4cef-8a67-0ec2f17f5643 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:03.628346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:03.657572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08570ffc-b932-4cef-8a67-0ec2f17f5643 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:03.712220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1229fb-5c97-436c-b101-fdc759099305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:03.712874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1229fb-5c97-436c-b101-fdc759099305 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:03.713181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1229fb-5c97-436c-b101-fdc759099305 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:03.713462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1229fb-5c97-436c-b101-fdc759099305 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:03.733970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1229fb-5c97-436c-b101-fdc759099305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-6f1229fb-5c97-436c-b101-fdc759099305 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:03.740913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:03.741841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:03.742634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 282/546] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:03.747891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cb5c9-bcff-4015-9aa9-9fea02c625ef None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:03.763824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1229fb-5c97-436c-b101-fdc759099305 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:03.873031 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cb5c9-bcff-4015-9aa9-9fea02c625ef None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:03.882075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:03.914402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eae8dd-8759-4259-aa76-390a30d6b819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:03.921113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eae8dd-8759-4259-aa76-390a30d6b819 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:03.921113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eae8dd-8759-4259-aa76-390a30d6b819 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:03.921113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eae8dd-8759-4259-aa76-390a30d6b819 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:03.921113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eae8dd-8759-4259-aa76-390a30d6b819 None admin] RBAC: Adding query filter params (name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:03.921113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eae8dd-8759-4259-aa76-390a30d6b819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:03.921113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eae8dd-8759-4259-aa76-390a30d6b819 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:03.927674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 265/547] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:03.932926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eae8dd-8759-4259-aa76-390a30d6b819 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:04.023075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1229fb-5c97-436c-b101-fdc759099305 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:04.049976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eae8dd-8759-4259-aa76-390a30d6b819 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:04.094350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949598f-eaf3-4a7f-9c23-2e7d6c939ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:04.095010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949598f-eaf3-4a7f-9c23-2e7d6c939ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:04.096786 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949598f-eaf3-4a7f-9c23-2e7d6c939ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:04.097128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949598f-eaf3-4a7f-9c23-2e7d6c939ba8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:04.115804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78373fc4-29a3-48e7-ac99-a98b87939f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:04.117965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78373fc4-29a3-48e7-ac99-a98b87939f46 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:04.118251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78373fc4-29a3-48e7-ac99-a98b87939f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:04.118523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78373fc4-29a3-48e7-ac99-a98b87939f46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:04.122359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949598f-eaf3-4a7f-9c23-2e7d6c939ba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-3949598f-eaf3-4a7f-9c23-2e7d6c939ba8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:04.134301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:04.135208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:04.136079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 283/548] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:04.141320 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78373fc4-29a3-48e7-ac99-a98b87939f46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:04.143477 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949598f-eaf3-4a7f-9c23-2e7d6c939ba8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:04.144497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78373fc4-29a3-48e7-ac99-a98b87939f46 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:04.149496 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-78373fc4-29a3-48e7-ac99-a98b87939f46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:04.183698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:04.186488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:04.189246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:04.189246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:04.189246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:04.192437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 266/549] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:04.203933 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78373fc4-29a3-48e7-ac99-a98b87939f46 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:04.244641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:04.359639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949598f-eaf3-4a7f-9c23-2e7d6c939ba8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:04.412235 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78373fc4-29a3-48e7-ac99-a98b87939f46 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:04.435019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d82c1a-b72c-4fa5-9bc8-3c697d14e8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:04.435642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d82c1a-b72c-4fa5-9bc8-3c697d14e8ae None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:04.435926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d82c1a-b72c-4fa5-9bc8-3c697d14e8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:04.436203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d82c1a-b72c-4fa5-9bc8-3c697d14e8ae None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:04.436881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d82c1a-b72c-4fa5-9bc8-3c697d14e8ae None admin] RBAC: Adding query filter params (name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:04.437221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d82c1a-b72c-4fa5-9bc8-3c697d14e8ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:04.439160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d82c1a-b72c-4fa5-9bc8-3c697d14e8ae None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:04.458949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 284/550] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:04.470661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d82c1a-b72c-4fa5-9bc8-3c697d14e8ae None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:04.489081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea1d0e-48d4-496f-9dc1-d606426a9340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:04.490086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea1d0e-48d4-496f-9dc1-d606426a9340 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:04.490566 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea1d0e-48d4-496f-9dc1-d606426a9340 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:04.491038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea1d0e-48d4-496f-9dc1-d606426a9340 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:04.491922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea1d0e-48d4-496f-9dc1-d606426a9340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:04.492425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea1d0e-48d4-496f-9dc1-d606426a9340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:04.494589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea1d0e-48d4-496f-9dc1-d606426a9340 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:04.535540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 267/551] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:04.542516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea1d0e-48d4-496f-9dc1-d606426a9340 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:04.625397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:04.692043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d82c1a-b72c-4fa5-9bc8-3c697d14e8ae None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:04.710131 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:04.730201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10bf0f-f527-43a1-9376-bd5f02609735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:04.731171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10bf0f-f527-43a1-9376-bd5f02609735 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:04.731676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10bf0f-f527-43a1-9376-bd5f02609735 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:04.732258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10bf0f-f527-43a1-9376-bd5f02609735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:04.735432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea1d0e-48d4-496f-9dc1-d606426a9340 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:04.743792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10bf0f-f527-43a1-9376-bd5f02609735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:04.745842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10bf0f-f527-43a1-9376-bd5f02609735 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-2f10bf0f-f527-43a1-9376-bd5f02609735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:04.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:04.761816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:04.762664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:04.762664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:04.762664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:04.764683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 285/552] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:04.766700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a5a9b-2b3f-40ff-8314-176569a8b5ba None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:04.767225 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a5a9b-2b3f-40ff-8314-176569a8b5ba None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:04.767711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a5a9b-2b3f-40ff-8314-176569a8b5ba None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:04.773132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 286/553] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:04.774153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a5a9b-2b3f-40ff-8314-176569a8b5ba None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:04.780493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b46998d-31d3-4081-8ec6-449aadcc22a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:04.781174 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46998d-31d3-4081-8ec6-449aadcc22a1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:04.781456 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46998d-31d3-4081-8ec6-449aadcc22a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:04.781791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46998d-31d3-4081-8ec6-449aadcc22a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:04.791447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b46998d-31d3-4081-8ec6-449aadcc22a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:04.794503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b46998d-31d3-4081-8ec6-449aadcc22a1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-1b46998d-31d3-4081-8ec6-449aadcc22a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:04.810024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:04.810945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:04.811918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:04.811918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:04.811918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:04.811918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:04.811918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:04.811918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:04.811918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:04.811918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:04.811918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 08:25:04.811918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:04.814388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 268/554] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:25:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:04.815918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187338cd-a823-4df0-b47b-8419f075bcae None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:04.816406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187338cd-a823-4df0-b47b-8419f075bcae None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:04.816881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187338cd-a823-4df0-b47b-8419f075bcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:04.841460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-187338cd-a823-4df0-b47b-8419f075bcae None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:04.842978 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-187338cd-a823-4df0-b47b-8419f075bcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:04.901997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a5a9b-2b3f-40ff-8314-176569a8b5ba None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:04.920144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:04.943362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e3497-e26a-44dd-9f84-a159ebf8f826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:04.943362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e3497-e26a-44dd-9f84-a159ebf8f826 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:04.943362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e3497-e26a-44dd-9f84-a159ebf8f826 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:04.943362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e3497-e26a-44dd-9f84-a159ebf8f826 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:04.943935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e3497-e26a-44dd-9f84-a159ebf8f826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:04.944325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e3497-e26a-44dd-9f84-a159ebf8f826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:04.944767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 269/555] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:04.945684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e3497-e26a-44dd-9f84-a159ebf8f826 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:04.946287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187338cd-a823-4df0-b47b-8419f075bcae None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:04.958965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 287/556] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:04.966630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e3497-e26a-44dd-9f84-a159ebf8f826 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:05.091930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:05.153546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187338cd-a823-4df0-b47b-8419f075bcae None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:05.158647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e3497-e26a-44dd-9f84-a159ebf8f826 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:05.223928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1c93a-6309-4024-be24-58020d371955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:05.224548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1c93a-6309-4024-be24-58020d371955 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:05.224824 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1c93a-6309-4024-be24-58020d371955 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:05.225109 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1c93a-6309-4024-be24-58020d371955 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:05.225856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1c93a-6309-4024-be24-58020d371955 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:05.226157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1c93a-6309-4024-be24-58020d371955 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:05.227591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1c93a-6309-4024-be24-58020d371955 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:05.232776 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1569e39-fca4-4d90-b2ef-b749f9e689a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:05.233435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1569e39-fca4-4d90-b2ef-b749f9e689a2 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:05.233708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1569e39-fca4-4d90-b2ef-b749f9e689a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:05.233971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1569e39-fca4-4d90-b2ef-b749f9e689a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:05.243295 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1569e39-fca4-4d90-b2ef-b749f9e689a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:05.244748 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1569e39-fca4-4d90-b2ef-b749f9e689a2 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-c1569e39-fca4-4d90-b2ef-b749f9e689a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:05.257041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:05.258499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:05.259415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:05.259415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:05.259415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:05.259415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:05.259415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:05.259415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:05.259415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:05.259415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:05.259415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:25:05.259415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:05.259415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 288/557] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 08:25:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:05.263845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1569e39-fca4-4d90-b2ef-b749f9e689a2 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:05.279367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 270/558] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 08:25:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:05.285376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1c93a-6309-4024-be24-58020d371955 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:05.300969 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:05.374729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1569e39-fca4-4d90-b2ef-b749f9e689a2 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:05.420835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932f171-1092-4629-874f-88b2a7dcaef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:05.421739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932f171-1092-4629-874f-88b2a7dcaef9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:05.422025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932f171-1092-4629-874f-88b2a7dcaef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:05.422338 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932f171-1092-4629-874f-88b2a7dcaef9 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:05.423106 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932f171-1092-4629-874f-88b2a7dcaef9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:05.423550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932f171-1092-4629-874f-88b2a7dcaef9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:05.425196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932f171-1092-4629-874f-88b2a7dcaef9 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:05.433576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1c93a-6309-4024-be24-58020d371955 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:05.447938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 289/559] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 08:25:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:05.448800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932f171-1092-4629-874f-88b2a7dcaef9 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:05.468403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:05.479873 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee504c-4341-4883-91f5-639fd6be19bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:05.480446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee504c-4341-4883-91f5-639fd6be19bb None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:05.480963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee504c-4341-4883-91f5-639fd6be19bb None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:05.481276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee504c-4341-4883-91f5-639fd6be19bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:05.577170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:05.587862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932f171-1092-4629-874f-88b2a7dcaef9 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:05.595863 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee504c-4341-4883-91f5-639fd6be19bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:05.597358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee504c-4341-4883-91f5-639fd6be19bb None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:05.631156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ea134-bddc-4ba3-9cc6-9f18f5956545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:05.632282 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ea134-bddc-4ba3-9cc6-9f18f5956545 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:05.633067 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ea134-bddc-4ba3-9cc6-9f18f5956545 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:05.633776 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ea134-bddc-4ba3-9cc6-9f18f5956545 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:05.646259 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ea134-bddc-4ba3-9cc6-9f18f5956545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-b13ea134-bddc-4ba3-9cc6-9f18f5956545 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:05.651350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:05.652455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:05.653395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 290/560] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:05.654092 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ea134-bddc-4ba3-9cc6-9f18f5956545 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:05.710568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 271/561] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:25:05.711951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2d25a-77b6-4e2f-9040-5c51d2e8247d None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:05.712247 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2d25a-77b6-4e2f-9040-5c51d2e8247d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:05.712520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2d25a-77b6-4e2f-9040-5c51d2e8247d None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:05.713856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 272/562] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:05.714868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad318fb-78ed-45ba-a4fb-78e810f1fad1 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:05.715156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad318fb-78ed-45ba-a4fb-78e810f1fad1 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:05.715431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad318fb-78ed-45ba-a4fb-78e810f1fad1 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:05.716862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 273/563] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:05.717825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad318fb-78ed-45ba-a4fb-78e810f1fad1 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:05.746209 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:05.762925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ea134-bddc-4ba3-9cc6-9f18f5956545 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:05.797422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db69f8b-0d60-4818-aafd-e3d3a71dac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:05.798077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db69f8b-0d60-4818-aafd-e3d3a71dac55 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:05.798352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db69f8b-0d60-4818-aafd-e3d3a71dac55 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:05.798624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db69f8b-0d60-4818-aafd-e3d3a71dac55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:05.808554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db69f8b-0d60-4818-aafd-e3d3a71dac55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:05.810018 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db69f8b-0d60-4818-aafd-e3d3a71dac55 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:05.824345 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad318fb-78ed-45ba-a4fb-78e810f1fad1 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-0db69f8b-0d60-4818-aafd-e3d3a71dac55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:05.825184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:05.826252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:05.827237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:05.827237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:05.827237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:05.827237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 291/564] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:05.827237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db69f8b-0d60-4818-aafd-e3d3a71dac55 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:05.837134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:05.866236 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11426b8f-6e23-41c0-82c5-79691eb9729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:05.866871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11426b8f-6e23-41c0-82c5-79691eb9729f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:05.867142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11426b8f-6e23-41c0-82c5-79691eb9729f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:05.867410 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11426b8f-6e23-41c0-82c5-79691eb9729f None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:05.868085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11426b8f-6e23-41c0-82c5-79691eb9729f None admin] RBAC: Adding query filter params (name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:05.868388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11426b8f-6e23-41c0-82c5-79691eb9729f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:05.869802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11426b8f-6e23-41c0-82c5-79691eb9729f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:05.880190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 274/565] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:05.881543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b644185-a4d9-495e-abb5-fb9c6584a12d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:05.881838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b644185-a4d9-495e-abb5-fb9c6584a12d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:05.882109 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b644185-a4d9-495e-abb5-fb9c6584a12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:05.900682 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-5b644185-a4d9-495e-abb5-fb9c6584a12d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:05.902231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5b644185-a4d9-495e-abb5-fb9c6584a12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:05.941032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db69f8b-0d60-4818-aafd-e3d3a71dac55 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:05.969297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 275/566] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:05.970543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1369743-6611-43e4-b6e0-71e5f2e97095 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:05.970825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1369743-6611-43e4-b6e0-71e5f2e97095 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:05.971099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1369743-6611-43e4-b6e0-71e5f2e97095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:05.976938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b78d6-e794-4b1b-9f35-c93b5b8b8fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:05.977663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b78d6-e794-4b1b-9f35-c93b5b8b8fda None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:05.978710 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b78d6-e794-4b1b-9f35-c93b5b8b8fda None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:05.978992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b78d6-e794-4b1b-9f35-c93b5b8b8fda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:05.988449 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-e1369743-6611-43e4-b6e0-71e5f2e97095 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:05.989678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e1369743-6611-43e4-b6e0-71e5f2e97095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:05.996815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b78d6-e794-4b1b-9f35-c93b5b8b8fda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:05.998248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b78d6-e794-4b1b-9f35-c93b5b8b8fda None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:06.006167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-ea9b78d6-e794-4b1b-9f35-c93b5b8b8fda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:06.013767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:06.014841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:06.015697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:06.015697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:06.015697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:06.015697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 292/567] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:06.015697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b78d6-e794-4b1b-9f35-c93b5b8b8fda None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:06.077353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 276/568] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:06.077353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1369743-6611-43e4-b6e0-71e5f2e97095 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:06.093444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:06.152725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b78d6-e794-4b1b-9f35-c93b5b8b8fda None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:06.190530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e882eba-a172-49c6-9046-d49067e5f82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:06.191989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e882eba-a172-49c6-9046-d49067e5f82a None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:06.192736 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e882eba-a172-49c6-9046-d49067e5f82a None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:06.193217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e882eba-a172-49c6-9046-d49067e5f82a None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:06.194268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e882eba-a172-49c6-9046-d49067e5f82a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:06.194722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e882eba-a172-49c6-9046-d49067e5f82a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:06.196227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e882eba-a172-49c6-9046-d49067e5f82a None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:06.199317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1369743-6611-43e4-b6e0-71e5f2e97095 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:06.216596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 293/569] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:06.217761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e882eba-a172-49c6-9046-d49067e5f82a None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:06.248750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5661d-605b-4402-b284-8eaaa5678e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:06.249581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5661d-605b-4402-b284-8eaaa5678e06 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:06.249860 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5661d-605b-4402-b284-8eaaa5678e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:06.250298 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5661d-605b-4402-b284-8eaaa5678e06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:06.267897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5661d-605b-4402-b284-8eaaa5678e06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:06.269744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5661d-605b-4402-b284-8eaaa5678e06 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-f3e5661d-605b-4402-b284-8eaaa5678e06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:06.290426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:06.291489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:06.292425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:06.292425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:06.292425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:06.293242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 277/570] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:06.294063 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5661d-605b-4402-b284-8eaaa5678e06 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:06.299321 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:06.333264 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e882eba-a172-49c6-9046-d49067e5f82a None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:06.350533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:06.368601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570b435-3636-4704-953a-31443c0185ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:06.369259 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570b435-3636-4704-953a-31443c0185ff None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:06.369545 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570b435-3636-4704-953a-31443c0185ff None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:06.369817 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570b435-3636-4704-953a-31443c0185ff None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:06.370460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570b435-3636-4704-953a-31443c0185ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:06.370766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570b435-3636-4704-953a-31443c0185ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:06.372146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570b435-3636-4704-953a-31443c0185ff None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:06.384340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 294/571] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:06.385383 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570b435-3636-4704-953a-31443c0185ff None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:06.406399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5661d-605b-4402-b284-8eaaa5678e06 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:06.444488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3e4db-2b27-404a-94bd-eb680df7b835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:06.445145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3e4db-2b27-404a-94bd-eb680df7b835 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:06.445425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3e4db-2b27-404a-94bd-eb680df7b835 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:06.445695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3e4db-2b27-404a-94bd-eb680df7b835 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:06.452419 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3e4db-2b27-404a-94bd-eb680df7b835 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:06.454480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3e4db-2b27-404a-94bd-eb680df7b835 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-53d3e4db-2b27-404a-94bd-eb680df7b835 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:06.465798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:06.466707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:06.467587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:06.467587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:06.467587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:06.467587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:06.467587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:06.467587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:06.467587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:06.467587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:06.467587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 278/572] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:06.468091 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3e4db-2b27-404a-94bd-eb680df7b835 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:06.494144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570b435-3636-4704-953a-31443c0185ff None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:06.530679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:06.531594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8a25b-375a-4f02-afa6-6f7117b80f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:06.532629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8a25b-375a-4f02-afa6-6f7117b80f30 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:06.533072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8a25b-375a-4f02-afa6-6f7117b80f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:06.533409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8a25b-375a-4f02-afa6-6f7117b80f30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:06.543692 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8a25b-375a-4f02-afa6-6f7117b80f30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-6af8a25b-375a-4f02-afa6-6f7117b80f30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:06.549579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:06.550643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 295/573] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:06.551590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8a25b-375a-4f02-afa6-6f7117b80f30 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:06.574721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3e4db-2b27-404a-94bd-eb680df7b835 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:06.593474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:06.629029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6badd-8baa-4e50-a4b6-7917cbb133eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:06.629673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6badd-8baa-4e50-a4b6-7917cbb133eb None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:06.629952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6badd-8baa-4e50-a4b6-7917cbb133eb None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:06.630228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca6badd-8baa-4e50-a4b6-7917cbb133eb None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:06.630868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca6badd-8baa-4e50-a4b6-7917cbb133eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:06.631170 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca6badd-8baa-4e50-a4b6-7917cbb133eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:06.632609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca6badd-8baa-4e50-a4b6-7917cbb133eb None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:06.646165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 279/574] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:06.646747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6badd-8baa-4e50-a4b6-7917cbb133eb None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:06.676182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8a25b-375a-4f02-afa6-6f7117b80f30 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:06.712079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e4f38-a8db-4a38-8a2a-d243cb6a2c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:06.712712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e4f38-a8db-4a38-8a2a-d243cb6a2c3f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:06.712989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e4f38-a8db-4a38-8a2a-d243cb6a2c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:06.713292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e4f38-a8db-4a38-8a2a-d243cb6a2c3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:06.728570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e4f38-a8db-4a38-8a2a-d243cb6a2c3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-680e4f38-a8db-4a38-8a2a-d243cb6a2c3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:06.733915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:06.735931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 296/575] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:06.737187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e4f38-a8db-4a38-8a2a-d243cb6a2c3f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:06.763629 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca6badd-8baa-4e50-a4b6-7917cbb133eb None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:06.790392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:06.802463 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab11e1-6b62-46f8-a194-99eadf1875b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:06.803093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab11e1-6b62-46f8-a194-99eadf1875b3 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:06.803369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab11e1-6b62-46f8-a194-99eadf1875b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:06.803666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab11e1-6b62-46f8-a194-99eadf1875b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:06.810577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab11e1-6b62-46f8-a194-99eadf1875b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:06.812245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab11e1-6b62-46f8-a194-99eadf1875b3 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-03ab11e1-6b62-46f8-a194-99eadf1875b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:06.824370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:06.825774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:06.826732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:06.826732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:06.826732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:06.826732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:06.826732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:06.826732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:06.826732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:06.826732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:06.826732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 280/576] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:06.826732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab11e1-6b62-46f8-a194-99eadf1875b3 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:06.857607 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:06.863173 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e4f38-a8db-4a38-8a2a-d243cb6a2c3f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:06.899280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0f188-04f1-44fe-8434-fe54cf04e073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:06.901453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0f188-04f1-44fe-8434-fe54cf04e073 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:06.901728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0f188-04f1-44fe-8434-fe54cf04e073 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:06.901995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0f188-04f1-44fe-8434-fe54cf04e073 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:06.902676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0f188-04f1-44fe-8434-fe54cf04e073 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:06.902976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0f188-04f1-44fe-8434-fe54cf04e073 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:06.904405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0f188-04f1-44fe-8434-fe54cf04e073 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:06.917029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 297/577] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:06.918036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0f188-04f1-44fe-8434-fe54cf04e073 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:06.934168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab11e1-6b62-46f8-a194-99eadf1875b3 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:06.972783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a89a76-3ccf-47ed-8084-b4e2002bf97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:06.973429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a89a76-3ccf-47ed-8084-b4e2002bf97f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:06.973711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a89a76-3ccf-47ed-8084-b4e2002bf97f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:06.973984 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a89a76-3ccf-47ed-8084-b4e2002bf97f None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:06.974645 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a89a76-3ccf-47ed-8084-b4e2002bf97f None admin] RBAC: Adding query filter params (name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:06.974951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a89a76-3ccf-47ed-8084-b4e2002bf97f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:06.976323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a89a76-3ccf-47ed-8084-b4e2002bf97f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:06.984613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 281/578] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:06.985964 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a89a76-3ccf-47ed-8084-b4e2002bf97f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:07.032261 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:07.041054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0f188-04f1-44fe-8434-fe54cf04e073 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:07.082183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa213b-e913-412e-8f5d-37d52ecc4388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:07.082852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa213b-e913-412e-8f5d-37d52ecc4388 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:07.083134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa213b-e913-412e-8f5d-37d52ecc4388 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:07.083408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa213b-e913-412e-8f5d-37d52ecc4388 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:07.090727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa213b-e913-412e-8f5d-37d52ecc4388 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:07.090932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a89a76-3ccf-47ed-8084-b4e2002bf97f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:07.092156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa213b-e913-412e-8f5d-37d52ecc4388 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-d7aa213b-e913-412e-8f5d-37d52ecc4388 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:07.109368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:07.111026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:07.111989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:07.111989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:07.111989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:07.111989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:07.111989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:07.111989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:07.111989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:07.111989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:07.111989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 08:25:07.111989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:07.111989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 298/579] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:25:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:07.111989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa213b-e913-412e-8f5d-37d52ecc4388 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:07.132512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2de8f-3b52-4f75-8586-60dfc41f2018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:07.133158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2de8f-3b52-4f75-8586-60dfc41f2018 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:07.133438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2de8f-3b52-4f75-8586-60dfc41f2018 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:07.133712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2de8f-3b52-4f75-8586-60dfc41f2018 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:07.134387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2de8f-3b52-4f75-8586-60dfc41f2018 None admin] RBAC: Adding query filter params (name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:07.134684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2de8f-3b52-4f75-8586-60dfc41f2018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:07.136050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2de8f-3b52-4f75-8586-60dfc41f2018 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:07.136722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:07.145488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 282/580] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:07.146439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2de8f-3b52-4f75-8586-60dfc41f2018 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:07.230488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa213b-e913-412e-8f5d-37d52ecc4388 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:07.267799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2de8f-3b52-4f75-8586-60dfc41f2018 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:07.268909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0338e54-c878-4be0-a031-d6f2183ad7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:07.269552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0338e54-c878-4be0-a031-d6f2183ad7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:07.269830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0338e54-c878-4be0-a031-d6f2183ad7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:07.270106 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0338e54-c878-4be0-a031-d6f2183ad7f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:07.270796 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0338e54-c878-4be0-a031-d6f2183ad7f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:07.271099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0338e54-c878-4be0-a031-d6f2183ad7f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:07.272519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0338e54-c878-4be0-a031-d6f2183ad7f5 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:07.275553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:07.285806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 299/581] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:07.286635 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0338e54-c878-4be0-a031-d6f2183ad7f5 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:07.309690 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129c2f4-628c-4a4f-b68e-fc20d8614452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:07.310369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129c2f4-628c-4a4f-b68e-fc20d8614452 None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:07.310647 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129c2f4-628c-4a4f-b68e-fc20d8614452 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:07.310912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129c2f4-628c-4a4f-b68e-fc20d8614452 None admin] PATH_INFO: `/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/9c3c6efc3c154ff89066035cd9583459/roles/5354696222554ab2a19a3a839682a597` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:07.325751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129c2f4-628c-4a4f-b68e-fc20d8614452 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca4d4801f2f40419f6881984fbbfbcd, user_id=9c3c6efc3c154ff89066035cd9583459, role_id=5354696222554ab2a19a3a839682a597)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:07.327184 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129c2f4-628c-4a4f-b68e-fc20d8614452 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:07.339908 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 283/582] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:25:07 2022] PUT /identity/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/9c3c6efc3c154ff89066035cd9583459/roles/5354696222554ab2a19a3a839682a597 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:25:07.340729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129c2f4-628c-4a4f-b68e-fc20d8614452 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:07.368046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:07.395473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0338e54-c878-4be0-a031-d6f2183ad7f5 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:07.438250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8457e-8932-4580-a859-e486fc630a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:07.439499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8457e-8932-4580-a859-e486fc630a74 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:07.441053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8457e-8932-4580-a859-e486fc630a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:07.441747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8457e-8932-4580-a859-e486fc630a74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:07.455857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8457e-8932-4580-a859-e486fc630a74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:07.457377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8457e-8932-4580-a859-e486fc630a74 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-b1c8457e-8932-4580-a859-e486fc630a74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:07.476314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:07.477404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:07.478281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:07.478281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:07.478281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:07.478281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 300/583] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:07.478509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8457e-8932-4580-a859-e486fc630a74 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:07.485808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129c2f4-628c-4a4f-b68e-fc20d8614452 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:07.539498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbefe04-fde9-4bcc-8d64-b8ea539561e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:07.540226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbefe04-fde9-4bcc-8d64-b8ea539561e1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:07.540508 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbefe04-fde9-4bcc-8d64-b8ea539561e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:07.540781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbefe04-fde9-4bcc-8d64-b8ea539561e1 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:07.541859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbefe04-fde9-4bcc-8d64-b8ea539561e1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:07.541859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbefe04-fde9-4bcc-8d64-b8ea539561e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:07.543379 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbefe04-fde9-4bcc-8d64-b8ea539561e1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:07.563198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 284/584] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 08:25:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:07.564365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbefe04-fde9-4bcc-8d64-b8ea539561e1 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:07.586277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:07.605825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8457e-8932-4580-a859-e486fc630a74 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:07.636095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:07.643797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb62569-ea5a-4ebd-96ff-cbeba3ed7336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:07.644433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb62569-ea5a-4ebd-96ff-cbeba3ed7336 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:07.644712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb62569-ea5a-4ebd-96ff-cbeba3ed7336 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:07.644984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb62569-ea5a-4ebd-96ff-cbeba3ed7336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:07.657982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb62569-ea5a-4ebd-96ff-cbeba3ed7336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:07.659485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb62569-ea5a-4ebd-96ff-cbeba3ed7336 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-1eb62569-ea5a-4ebd-96ff-cbeba3ed7336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:07.674806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:07.675960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:07.677138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:07.677138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:07.677138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:07.680698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 301/585] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:07.681381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbefe04-fde9-4bcc-8d64-b8ea539561e1 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:07.681598 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb62569-ea5a-4ebd-96ff-cbeba3ed7336 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:07.718284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee7147-d914-4183-aa3e-729e909c0e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:07.718895 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee7147-d914-4183-aa3e-729e909c0e62 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:07.719220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee7147-d914-4183-aa3e-729e909c0e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:07.720091 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee7147-d914-4183-aa3e-729e909c0e62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:07.720812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee7147-d914-4183-aa3e-729e909c0e62 None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=2f410e0148c1403098b9b73735425726) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:07.721150 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee7147-d914-4183-aa3e-729e909c0e62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:07.722571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee7147-d914-4183-aa3e-729e909c0e62 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:07.728482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 285/586] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:07 2022] GET /identity/v3/role_assignments?user.id=2f410e0148c1403098b9b73735425726&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:07.729970 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee7147-d914-4183-aa3e-729e909c0e62 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:07.829556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb62569-ea5a-4ebd-96ff-cbeba3ed7336 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:07.836641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:07.872451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee7147-d914-4183-aa3e-729e909c0e62 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:07.879721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d960281-9d2f-4c6e-bd67-c400fc922b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:07.880463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d960281-9d2f-4c6e-bd67-c400fc922b29 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:07.880815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d960281-9d2f-4c6e-bd67-c400fc922b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:07.881269 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d960281-9d2f-4c6e-bd67-c400fc922b29 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:07.882012 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d960281-9d2f-4c6e-bd67-c400fc922b29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:07.882455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d960281-9d2f-4c6e-bd67-c400fc922b29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:07.883964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d960281-9d2f-4c6e-bd67-c400fc922b29 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:07.899600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 302/587] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:07.900493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d960281-9d2f-4c6e-bd67-c400fc922b29 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:07.928570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01449da6-33df-4437-b121-fde6c73c31e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:07.929363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01449da6-33df-4437-b121-fde6c73c31e0 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:07.929711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01449da6-33df-4437-b121-fde6c73c31e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:07.930056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01449da6-33df-4437-b121-fde6c73c31e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:07.943790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01449da6-33df-4437-b121-fde6c73c31e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:07.945451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01449da6-33df-4437-b121-fde6c73c31e0 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-01449da6-33df-4437-b121-fde6c73c31e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:07.964631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:07.966264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:07.967317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:07.967317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:07.967317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:07.973123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 286/588] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:07.973123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01449da6-33df-4437-b121-fde6c73c31e0 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:07.979760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:08.056647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d960281-9d2f-4c6e-bd67-c400fc922b29 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:08.097749 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01449da6-33df-4437-b121-fde6c73c31e0 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:08.099606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a9008-a1ae-466a-b790-835625385fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:08.100389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a9008-a1ae-466a-b790-835625385fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:08.101134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a9008-a1ae-466a-b790-835625385fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:08.101134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a9008-a1ae-466a-b790-835625385fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:08.101968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a9008-a1ae-466a-b790-835625385fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:08.102301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a9008-a1ae-466a-b790-835625385fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:08.103756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a9008-a1ae-466a-b790-835625385fd1 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:08.120743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 303/589] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:08.120743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a9008-a1ae-466a-b790-835625385fd1 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:08.145200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:08.154154 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddee74-184b-459e-a3ec-0cb027b1f4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:08.155756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddee74-184b-459e-a3ec-0cb027b1f4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:08.156215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddee74-184b-459e-a3ec-0cb027b1f4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:08.156678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddee74-184b-459e-a3ec-0cb027b1f4d9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:08.166122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddee74-184b-459e-a3ec-0cb027b1f4d9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:08.167851 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddee74-184b-459e-a3ec-0cb027b1f4d9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-e5ddee74-184b-459e-a3ec-0cb027b1f4d9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:08.179921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:08.180995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:08.181959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:08.181959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:08.181959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:08.181959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:08.181959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:08.181959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:08.181959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:08.181959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:08.181959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 08:25:08.181959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:08.185924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 287/590] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:08.186837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddee74-184b-459e-a3ec-0cb027b1f4d9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:08.277916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a9008-a1ae-466a-b790-835625385fd1 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:08.280975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:08.305269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddee74-184b-459e-a3ec-0cb027b1f4d9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:08.332800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c601b-5b36-43be-b9d5-a658a487085d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:08.333622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c601b-5b36-43be-b9d5-a658a487085d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:08.333914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c601b-5b36-43be-b9d5-a658a487085d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:08.334203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c601b-5b36-43be-b9d5-a658a487085d None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:08.335462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c601b-5b36-43be-b9d5-a658a487085d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:08.335775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c601b-5b36-43be-b9d5-a658a487085d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:08.337626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c601b-5b36-43be-b9d5-a658a487085d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:08.345443 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfc2c1-08ed-4b8b-8139-266f0403b58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:08.346080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfc2c1-08ed-4b8b-8139-266f0403b58f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:08.346361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfc2c1-08ed-4b8b-8139-266f0403b58f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:08.346635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfc2c1-08ed-4b8b-8139-266f0403b58f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:08.353113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 304/591] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:08.353705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c601b-5b36-43be-b9d5-a658a487085d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:08.360394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbfc2c1-08ed-4b8b-8139-266f0403b58f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:08.362008 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbfc2c1-08ed-4b8b-8139-266f0403b58f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-3bbfc2c1-08ed-4b8b-8139-266f0403b58f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:08.377408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:08.378418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:08.379955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:08.379955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:08.379955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:08.379955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:08.379955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:08.379955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:08.379955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:08.379955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:08.379955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 08:25:08.379955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:08.379955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 288/592] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:25:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:08.379955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfc2c1-08ed-4b8b-8139-266f0403b58f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:08.441861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:08.481520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c601b-5b36-43be-b9d5-a658a487085d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:08.500133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfc2c1-08ed-4b8b-8139-266f0403b58f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:08.528612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e5d20-0c7a-4b5f-bea9-3eaa674b73ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:08.529410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e5d20-0c7a-4b5f-bea9-3eaa674b73ba None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:08.529683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e5d20-0c7a-4b5f-bea9-3eaa674b73ba None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:08.529946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e5d20-0c7a-4b5f-bea9-3eaa674b73ba None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:08.530704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e5d20-0c7a-4b5f-bea9-3eaa674b73ba None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:08.530957 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e5d20-0c7a-4b5f-bea9-3eaa674b73ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:08.532608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e5d20-0c7a-4b5f-bea9-3eaa674b73ba None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:08.537032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe93582-fd59-4421-8185-0fa6713d9949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:08.537697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe93582-fd59-4421-8185-0fa6713d9949 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:08.538019 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe93582-fd59-4421-8185-0fa6713d9949 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:08.538292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe93582-fd59-4421-8185-0fa6713d9949 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:08.545543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe93582-fd59-4421-8185-0fa6713d9949 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:08.547149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe93582-fd59-4421-8185-0fa6713d9949 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:08.551509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 305/593] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Sep 28 08:25:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:08.552357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e5d20-0c7a-4b5f-bea9-3eaa674b73ba None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-9fe93582-fd59-4421-8185-0fa6713d9949 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:08.560377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:08.562646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:08.565356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:08.565356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:08.565356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:08.565356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:08.565356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:08.565356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:08.565356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:08.565356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:08.565356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 289/594] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:08.565356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe93582-fd59-4421-8185-0fa6713d9949 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:08.576037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:08.662665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e5d20-0c7a-4b5f-bea9-3eaa674b73ba None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:08.675153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe93582-fd59-4421-8185-0fa6713d9949 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:08.686677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:08.701438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00615b87-de5f-4279-9476-e77c79b44f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:08.702263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00615b87-de5f-4279-9476-e77c79b44f18 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:08.702684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00615b87-de5f-4279-9476-e77c79b44f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:08.703124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00615b87-de5f-4279-9476-e77c79b44f18 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:08.704717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00615b87-de5f-4279-9476-e77c79b44f18 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:08.705256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00615b87-de5f-4279-9476-e77c79b44f18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:08.706884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00615b87-de5f-4279-9476-e77c79b44f18 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:08.712854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a271c-7edd-4cb7-a080-cd46e96d47b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:08.713520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a271c-7edd-4cb7-a080-cd46e96d47b2 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:08.713798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a271c-7edd-4cb7-a080-cd46e96d47b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:08.714072 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a271c-7edd-4cb7-a080-cd46e96d47b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:08.725676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a271c-7edd-4cb7-a080-cd46e96d47b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:08.727238 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a271c-7edd-4cb7-a080-cd46e96d47b2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:08.727397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 306/595] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 08:25:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:08.728380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00615b87-de5f-4279-9476-e77c79b44f18 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-883a271c-7edd-4cb7-a080-cd46e96d47b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:08.742741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:08.743840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:08.744751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:08.744751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:08.744751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:08.744751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 290/596] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:08.744992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a271c-7edd-4cb7-a080-cd46e96d47b2 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:08.820877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:08.847207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00615b87-de5f-4279-9476-e77c79b44f18 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:08.857845 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a271c-7edd-4cb7-a080-cd46e96d47b2 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:08.885108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a8a2d-d11e-4111-a0b8-d3297348ea64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:08.885857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a8a2d-d11e-4111-a0b8-d3297348ea64 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:08.886351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a8a2d-d11e-4111-a0b8-d3297348ea64 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:08.886728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a8a2d-d11e-4111-a0b8-d3297348ea64 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:08.887621 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a8a2d-d11e-4111-a0b8-d3297348ea64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:08.888032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a8a2d-d11e-4111-a0b8-d3297348ea64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:08.889579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a8a2d-d11e-4111-a0b8-d3297348ea64 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:08.896295 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbe008-b303-46b3-8719-ae417d2e0727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:08.897135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbe008-b303-46b3-8719-ae417d2e0727 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:08.897558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbe008-b303-46b3-8719-ae417d2e0727 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:08.898081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbe008-b303-46b3-8719-ae417d2e0727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:08.904446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 307/597] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:08.905465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a8a2d-d11e-4111-a0b8-d3297348ea64 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:08.910531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fbe008-b303-46b3-8719-ae417d2e0727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:08.912245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fbe008-b303-46b3-8719-ae417d2e0727 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-39fbe008-b303-46b3-8719-ae417d2e0727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:08.927899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:08.928917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:08.929819 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:08.929819 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:08.929819 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:08.931982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 291/598] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:08.933028 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbe008-b303-46b3-8719-ae417d2e0727 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:08.950086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:09.021389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a8a2d-d11e-4111-a0b8-d3297348ea64 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:09.043838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbe008-b303-46b3-8719-ae417d2e0727 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:09.058627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25846d-7c78-4e46-a8b6-28ff9907ab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:09.059535 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25846d-7c78-4e46-a8b6-28ff9907ab2f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:09.059963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25846d-7c78-4e46-a8b6-28ff9907ab2f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:09.060436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25846d-7c78-4e46-a8b6-28ff9907ab2f None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:09.061301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25846d-7c78-4e46-a8b6-28ff9907ab2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:09.061900 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25846d-7c78-4e46-a8b6-28ff9907ab2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:09.064340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25846d-7c78-4e46-a8b6-28ff9907ab2f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:09.072548 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:09.079301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 308/599] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:09.080319 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25846d-7c78-4e46-a8b6-28ff9907ab2f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:09.083878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c985b-45ef-4519-9197-4414ea5f91fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:09.084825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c985b-45ef-4519-9197-4414ea5f91fb None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:09.085327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c985b-45ef-4519-9197-4414ea5f91fb None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:09.085782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c985b-45ef-4519-9197-4414ea5f91fb None admin] PATH_INFO: `/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/5274740e5d7c4ab1ba9e9781815532c9/roles/5354696222554ab2a19a3a839682a597` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:09.102370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c985b-45ef-4519-9197-4414ea5f91fb None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca4d4801f2f40419f6881984fbbfbcd, user_id=5274740e5d7c4ab1ba9e9781815532c9, role_id=5354696222554ab2a19a3a839682a597)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:09.104107 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c985b-45ef-4519-9197-4414ea5f91fb None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:09.121548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 292/600] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:25:08 2022] PUT /identity/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/5274740e5d7c4ab1ba9e9781815532c9/roles/5354696222554ab2a19a3a839682a597 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:25:09.122572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c985b-45ef-4519-9197-4414ea5f91fb None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:09.218049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25846d-7c78-4e46-a8b6-28ff9907ab2f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:09.288862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460e791-98c7-4d31-bf91-2a388f2bfb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:09.290577 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460e791-98c7-4d31-bf91-2a388f2bfb46 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:09.290863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460e791-98c7-4d31-bf91-2a388f2bfb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:09.291138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460e791-98c7-4d31-bf91-2a388f2bfb46 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:09.291810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460e791-98c7-4d31-bf91-2a388f2bfb46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:09.292117 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460e791-98c7-4d31-bf91-2a388f2bfb46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:09.293562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460e791-98c7-4d31-bf91-2a388f2bfb46 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:09.299693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:09.319370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 309/601] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:09.327100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460e791-98c7-4d31-bf91-2a388f2bfb46 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:09.364757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c985b-45ef-4519-9197-4414ea5f91fb None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:09.412216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:09.414066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32d7cb-8cea-42a9-a5a0-e93cbd300190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:09.414833 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32d7cb-8cea-42a9-a5a0-e93cbd300190 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:09.415191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32d7cb-8cea-42a9-a5a0-e93cbd300190 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:09.415636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32d7cb-8cea-42a9-a5a0-e93cbd300190 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:09.427447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32d7cb-8cea-42a9-a5a0-e93cbd300190 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:09.429449 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32d7cb-8cea-42a9-a5a0-e93cbd300190 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-ec32d7cb-8cea-42a9-a5a0-e93cbd300190 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:09.445006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:09.445951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:09.446814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:09.446814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:09.446814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:09.446814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:09.446814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:09.446814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:09.446814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:09.446814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:09.447234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 293/602] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:09.459906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32d7cb-8cea-42a9-a5a0-e93cbd300190 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:09.478952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460e791-98c7-4d31-bf91-2a388f2bfb46 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:09.545429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154fcf6-cb4c-43b6-9eb0-6d91c10446ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:09.546066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154fcf6-cb4c-43b6-9eb0-6d91c10446ce None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:09.546346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154fcf6-cb4c-43b6-9eb0-6d91c10446ce None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:09.546617 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154fcf6-cb4c-43b6-9eb0-6d91c10446ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:09.555492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154fcf6-cb4c-43b6-9eb0-6d91c10446ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:09.556946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154fcf6-cb4c-43b6-9eb0-6d91c10446ce None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-e154fcf6-cb4c-43b6-9eb0-6d91c10446ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:09.568870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:09.570602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:09.572004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:09.572004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:09.572004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:09.572004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:09.572004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:09.572004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:09.572004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:09.572004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:09.572004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 310/603] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:09.578085 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154fcf6-cb4c-43b6-9eb0-6d91c10446ce None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:09.680676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32d7cb-8cea-42a9-a5a0-e93cbd300190 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:09.731934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:09.749792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154fcf6-cb4c-43b6-9eb0-6d91c10446ce None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:09.762110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfdda0-9856-45b4-b68f-db0e8ad8f176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:09.762730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfdda0-9856-45b4-b68f-db0e8ad8f176 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:09.763010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfdda0-9856-45b4-b68f-db0e8ad8f176 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:09.763279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfdda0-9856-45b4-b68f-db0e8ad8f176 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:09.763975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfdda0-9856-45b4-b68f-db0e8ad8f176 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:09.764277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfdda0-9856-45b4-b68f-db0e8ad8f176 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:09.765739 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfdda0-9856-45b4-b68f-db0e8ad8f176 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:09.779651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 294/604] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:09.785556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfdda0-9856-45b4-b68f-db0e8ad8f176 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:09.794588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:09.808109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a3a65-6f92-4c5d-838a-30ad2f14a098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:09.808735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a3a65-6f92-4c5d-838a-30ad2f14a098 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:09.809016 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a3a65-6f92-4c5d-838a-30ad2f14a098 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:09.809355 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a3a65-6f92-4c5d-838a-30ad2f14a098 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:09.810153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a3a65-6f92-4c5d-838a-30ad2f14a098 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:09.810465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a3a65-6f92-4c5d-838a-30ad2f14a098 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:09.812447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a3a65-6f92-4c5d-838a-30ad2f14a098 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:09.827707 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 311/605] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:09.830368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3abc05-7aee-4fbd-82bd-30dc5d3a984d None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:09.830655 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3abc05-7aee-4fbd-82bd-30dc5d3a984d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:09.830927 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3abc05-7aee-4fbd-82bd-30dc5d3a984d None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:09.832410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 312/606] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:09.834375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3abc05-7aee-4fbd-82bd-30dc5d3a984d None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:09.940029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfdda0-9856-45b4-b68f-db0e8ad8f176 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:09.946750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3abc05-7aee-4fbd-82bd-30dc5d3a984d None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:09.984412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece61f8b-7b31-4b77-8c0c-f5f4f9df8c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:09.985535 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece61f8b-7b31-4b77-8c0c-f5f4f9df8c09 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:09.985810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece61f8b-7b31-4b77-8c0c-f5f4f9df8c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:09.986079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece61f8b-7b31-4b77-8c0c-f5f4f9df8c09 None admin] PATH_INFO: `/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/ff22c2632c614fca8008feab93c8fa2d/roles/5354696222554ab2a19a3a839682a597` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:09.987431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48b2fb-d4ab-4c58-ba3c-34eaf1377c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:09.988167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b2fb-d4ab-4c58-ba3c-34eaf1377c47 None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:09.988740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b2fb-d4ab-4c58-ba3c-34eaf1377c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:09.989083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48b2fb-d4ab-4c58-ba3c-34eaf1377c47 None admin] PATH_INFO: `/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/736efafe9fc94b45bc75566b1696c498/roles/5354696222554ab2a19a3a839682a597` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:10.004587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece61f8b-7b31-4b77-8c0c-f5f4f9df8c09 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca4d4801f2f40419f6881984fbbfbcd, user_id=ff22c2632c614fca8008feab93c8fa2d, role_id=5354696222554ab2a19a3a839682a597)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:10.006139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece61f8b-7b31-4b77-8c0c-f5f4f9df8c09 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:10.007552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48b2fb-d4ab-4c58-ba3c-34eaf1377c47 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca4d4801f2f40419f6881984fbbfbcd, user_id=736efafe9fc94b45bc75566b1696c498, role_id=5354696222554ab2a19a3a839682a597)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:10.010235 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48b2fb-d4ab-4c58-ba3c-34eaf1377c47 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:10.023819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 313/607] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:25:09 2022] PUT /identity/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/ff22c2632c614fca8008feab93c8fa2d/roles/5354696222554ab2a19a3a839682a597 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:25:10.025315 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce485e9a-5809-4492-b6ff-c3668b318ecb None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:10.025604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce485e9a-5809-4492-b6ff-c3668b318ecb None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:10.025877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce485e9a-5809-4492-b6ff-c3668b318ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:10.030322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 295/608] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:25:09 2022] PUT /identity/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/736efafe9fc94b45bc75566b1696c498/roles/5354696222554ab2a19a3a839682a597 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:25:10.052044 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-ce485e9a-5809-4492-b6ff-c3668b318ecb None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:10.053491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ce485e9a-5809-4492-b6ff-c3668b318ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:10.083895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:10.137800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 314/609] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:10.362943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9474c9-e6c4-4edb-ab79-15c7d8c3721a None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:10.363290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9474c9-e6c4-4edb-ab79-15c7d8c3721a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:10.363388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9474c9-e6c4-4edb-ab79-15c7d8c3721a None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:10.364980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 296/610] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:10.370343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6c5fc-3d3c-455f-ae7f-225ff58e33fd None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:10.370898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6c5fc-3d3c-455f-ae7f-225ff58e33fd None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:10.370898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6c5fc-3d3c-455f-ae7f-225ff58e33fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:10.393993 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-c2e6c5fc-3d3c-455f-ae7f-225ff58e33fd None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:10.395169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c2e6c5fc-3d3c-455f-ae7f-225ff58e33fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:10.486442 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 315/611] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:10.518677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9474c9-e6c4-4edb-ab79-15c7d8c3721a None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:10.572627 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:10.672052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9474c9-e6c4-4edb-ab79-15c7d8c3721a None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:10.720142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3137e9-1607-4e92-a906-5e901a2259be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:10.720779 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3137e9-1607-4e92-a906-5e901a2259be None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:10.721054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3137e9-1607-4e92-a906-5e901a2259be None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:10.721377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3137e9-1607-4e92-a906-5e901a2259be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:10.739208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3137e9-1607-4e92-a906-5e901a2259be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-6c3137e9-1607-4e92-a906-5e901a2259be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:10.743975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:10.746075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:10.748037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 297/612] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:10.753793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6c5fc-3d3c-455f-ae7f-225ff58e33fd None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:10.794693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3137e9-1607-4e92-a906-5e901a2259be None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:10.860115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:10.888869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6c5fc-3d3c-455f-ae7f-225ff58e33fd None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:10.932421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3137e9-1607-4e92-a906-5e901a2259be None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:10.935596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:10.972650 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092657d5-373b-4b83-b908-73e00bce2b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:10.973454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092657d5-373b-4b83-b908-73e00bce2b54 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:10.973627 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092657d5-373b-4b83-b908-73e00bce2b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:10.973823 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092657d5-373b-4b83-b908-73e00bce2b54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:10.984332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092657d5-373b-4b83-b908-73e00bce2b54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-092657d5-373b-4b83-b908-73e00bce2b54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:10.989165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:10.991228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:10.993175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 298/613] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:10.998152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092657d5-373b-4b83-b908-73e00bce2b54 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:11.005169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563842e-b8de-4ad7-a8a3-4bd6b9fefa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:11.005800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563842e-b8de-4ad7-a8a3-4bd6b9fefa66 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:11.006083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563842e-b8de-4ad7-a8a3-4bd6b9fefa66 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:11.006358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563842e-b8de-4ad7-a8a3-4bd6b9fefa66 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:11.007037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563842e-b8de-4ad7-a8a3-4bd6b9fefa66 None admin] RBAC: Adding query filter params (name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:11.007342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563842e-b8de-4ad7-a8a3-4bd6b9fefa66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:11.008707 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563842e-b8de-4ad7-a8a3-4bd6b9fefa66 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:11.032855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 316/614] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:11.034157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563842e-b8de-4ad7-a8a3-4bd6b9fefa66 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:11.200699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092657d5-373b-4b83-b908-73e00bce2b54 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:11.224389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563842e-b8de-4ad7-a8a3-4bd6b9fefa66 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:11.265644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31157e42-2a11-48ae-a1c1-241b56e077fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:11.266326 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31157e42-2a11-48ae-a1c1-241b56e077fd None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:11.266606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31157e42-2a11-48ae-a1c1-241b56e077fd None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:11.266881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31157e42-2a11-48ae-a1c1-241b56e077fd None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:11.267572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31157e42-2a11-48ae-a1c1-241b56e077fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:11.267877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31157e42-2a11-48ae-a1c1-241b56e077fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:11.269297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31157e42-2a11-48ae-a1c1-241b56e077fd None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:11.287254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6baa2-c157-4b2b-a9f6-d4482c2432b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:11.287897 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6baa2-c157-4b2b-a9f6-d4482c2432b1 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:11.288179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6baa2-c157-4b2b-a9f6-d4482c2432b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:11.288451 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6baa2-c157-4b2b-a9f6-d4482c2432b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:11.296548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 299/615] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:11.298556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31157e42-2a11-48ae-a1c1-241b56e077fd None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:11.320612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6baa2-c157-4b2b-a9f6-d4482c2432b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:11.320612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6baa2-c157-4b2b-a9f6-d4482c2432b1 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:11.349460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-d5d6baa2-c157-4b2b-a9f6-d4482c2432b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:11.350853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:11.352519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:11.353417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:11.353417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:11.353417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:11.354696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 317/616] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:11.360413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6baa2-c157-4b2b-a9f6-d4482c2432b1 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:11.363529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:11.464331 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31157e42-2a11-48ae-a1c1-241b56e077fd None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:11.529882 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc890e-f588-4605-9b17-781a48514096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:11.530523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc890e-f588-4605-9b17-781a48514096 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:11.530801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc890e-f588-4605-9b17-781a48514096 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:11.531071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc890e-f588-4605-9b17-781a48514096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:11.546201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6baa2-c157-4b2b-a9f6-d4482c2432b1 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:11.550393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc890e-f588-4605-9b17-781a48514096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:11.555192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc890e-f588-4605-9b17-781a48514096 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-d7fc890e-f588-4605-9b17-781a48514096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:11.576692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:11.577716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:11.579137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:11.579137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:11.579137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:11.584110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 300/617] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:11.590578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc890e-f588-4605-9b17-781a48514096 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:11.615375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffe38a-4748-4d1f-8d6f-9b7199c1041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:11.615990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffe38a-4748-4d1f-8d6f-9b7199c1041c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:11.616263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffe38a-4748-4d1f-8d6f-9b7199c1041c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:11.616528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffe38a-4748-4d1f-8d6f-9b7199c1041c None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:11.617206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffe38a-4748-4d1f-8d6f-9b7199c1041c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:11.617624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffe38a-4748-4d1f-8d6f-9b7199c1041c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:11.618975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffe38a-4748-4d1f-8d6f-9b7199c1041c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:11.638063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 318/618] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:11.649476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffe38a-4748-4d1f-8d6f-9b7199c1041c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:11.697120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:11.718250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:11.752227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffe38a-4748-4d1f-8d6f-9b7199c1041c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:11.768626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc890e-f588-4605-9b17-781a48514096 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:11.812499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725597ba-186c-4736-bbee-ab659277f764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:11.813158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725597ba-186c-4736-bbee-ab659277f764 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:11.813438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725597ba-186c-4736-bbee-ab659277f764 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:11.813710 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725597ba-186c-4736-bbee-ab659277f764 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:11.814554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725597ba-186c-4736-bbee-ab659277f764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:11.814868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725597ba-186c-4736-bbee-ab659277f764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:11.816374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725597ba-186c-4736-bbee-ab659277f764 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:11.823219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702c372-6abf-4db7-b82d-05e72825ad35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:11.824588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702c372-6abf-4db7-b82d-05e72825ad35 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:11.825261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702c372-6abf-4db7-b82d-05e72825ad35 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:11.826168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702c372-6abf-4db7-b82d-05e72825ad35 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:11.837799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 301/619] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:11.842202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725597ba-186c-4736-bbee-ab659277f764 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:11.844022 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702c372-6abf-4db7-b82d-05e72825ad35 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:11.846393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702c372-6abf-4db7-b82d-05e72825ad35 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-e702c372-6abf-4db7-b82d-05e72825ad35 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:11.859569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:11.861168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:11.862121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:11.862121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:11.862121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:11.862121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:11.862121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:11.862121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:11.862121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:11.862121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:11.862121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:25:11.862121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:11.864974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 319/620] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 08:25:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:11.872303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702c372-6abf-4db7-b82d-05e72825ad35 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:11.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725597ba-186c-4736-bbee-ab659277f764 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:12.019401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:12.029641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b41dd0-3d41-4c25-a93f-b711fd7b9a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:12.030583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b41dd0-3d41-4c25-a93f-b711fd7b9a46 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.031046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b41dd0-3d41-4c25-a93f-b711fd7b9a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.031539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b41dd0-3d41-4c25-a93f-b711fd7b9a46 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.041680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b41dd0-3d41-4c25-a93f-b711fd7b9a46 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:12.043951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b41dd0-3d41-4c25-a93f-b711fd7b9a46 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:12.051376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702c372-6abf-4db7-b82d-05e72825ad35 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-99b41dd0-3d41-4c25-a93f-b711fd7b9a46 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:12.059782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:12.060765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:12.062427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:12.062427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:12.062427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:12.062427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:12.062427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:12.062427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:12.062427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:12.062427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:12.062427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 08:25:12.062427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:12.062427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 302/621] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:25:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:12.069555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e7333-85b0-4d4e-a6db-627ff736650b None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.069555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e7333-85b0-4d4e-a6db-627ff736650b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.069555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e7333-85b0-4d4e-a6db-627ff736650b None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.073766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 303/622] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:12.075910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e7333-85b0-4d4e-a6db-627ff736650b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:12.093478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:12.103012 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c11337-fb86-4150-bfaa-8e58b7a87b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:12.103630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c11337-fb86-4150-bfaa-8e58b7a87b93 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.103904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c11337-fb86-4150-bfaa-8e58b7a87b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.104224 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c11337-fb86-4150-bfaa-8e58b7a87b93 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.104956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c11337-fb86-4150-bfaa-8e58b7a87b93 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:12.105328 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c11337-fb86-4150-bfaa-8e58b7a87b93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:12.106751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c11337-fb86-4150-bfaa-8e58b7a87b93 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:12.128770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 320/623] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 08:25:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:12.130071 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af038c4f-5dff-47f9-8c05-ef9921b2607c None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.130351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af038c4f-5dff-47f9-8c05-ef9921b2607c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.130620 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af038c4f-5dff-47f9-8c05-ef9921b2607c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.150209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-af038c4f-5dff-47f9-8c05-ef9921b2607c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:12.151662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-af038c4f-5dff-47f9-8c05-ef9921b2607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:12.206511 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e7333-85b0-4d4e-a6db-627ff736650b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:12.237240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 321/624] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:12.238446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af038c4f-5dff-47f9-8c05-ef9921b2607c None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:12.254520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013dc2a-f33a-4507-b978-4a99bb0e2bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:12.255486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013dc2a-f33a-4507-b978-4a99bb0e2bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.257427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013dc2a-f33a-4507-b978-4a99bb0e2bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.257988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013dc2a-f33a-4507-b978-4a99bb0e2bd8 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.258922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013dc2a-f33a-4507-b978-4a99bb0e2bd8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:12.259668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013dc2a-f33a-4507-b978-4a99bb0e2bd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:12.263224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013dc2a-f33a-4507-b978-4a99bb0e2bd8 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:12.302620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 304/625] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 08:25:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:12.306886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013dc2a-f33a-4507-b978-4a99bb0e2bd8 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:12.389225 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:12.470247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af038c4f-5dff-47f9-8c05-ef9921b2607c None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:12.509559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:12.523190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013dc2a-f33a-4507-b978-4a99bb0e2bd8 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:12.570703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bae69-3aa2-40cb-8001-fb50179ba79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:12.572426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bae69-3aa2-40cb-8001-fb50179ba79a None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.572426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bae69-3aa2-40cb-8001-fb50179ba79a None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.572426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bae69-3aa2-40cb-8001-fb50179ba79a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.621115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059d102-0a29-427c-b563-0567bf7f2c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:12.621115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059d102-0a29-427c-b563-0567bf7f2c4c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.621115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059d102-0a29-427c-b563-0567bf7f2c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.621115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059d102-0a29-427c-b563-0567bf7f2c4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.633119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059d102-0a29-427c-b563-0567bf7f2c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:12.634862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059d102-0a29-427c-b563-0567bf7f2c4c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-1059d102-0a29-427c-b563-0567bf7f2c4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:12.656251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:12.657951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:12.659054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:12.659054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:12.659054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:12.660624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 305/626] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:12.661732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059d102-0a29-427c-b563-0567bf7f2c4c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:12.664271 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bae69-3aa2-40cb-8001-fb50179ba79a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:12.665952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bae69-3aa2-40cb-8001-fb50179ba79a None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:12.770558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:12.774970 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059d102-0a29-427c-b563-0567bf7f2c4c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:12.775704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 322/627] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:25:12.777894 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9983d-448a-4c51-8262-d5e76e1cab6c None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.779093 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9983d-448a-4c51-8262-d5e76e1cab6c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.779369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9983d-448a-4c51-8262-d5e76e1cab6c None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.785229 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 323/628] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:12.786600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a10e1-a28b-4c8d-b350-87065e337222 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.786881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a10e1-a28b-4c8d-b350-87065e337222 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.787187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a10e1-a28b-4c8d-b350-87065e337222 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.788550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 324/629] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:12.789790 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a10e1-a28b-4c8d-b350-87065e337222 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:12.806111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:12.819657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2a34a-e26f-4dd2-a299-ee4b0308558a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:12.820457 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2a34a-e26f-4dd2-a299-ee4b0308558a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.820790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2a34a-e26f-4dd2-a299-ee4b0308558a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.821011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2a34a-e26f-4dd2-a299-ee4b0308558a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.833307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2a34a-e26f-4dd2-a299-ee4b0308558a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-d5f2a34a-e26f-4dd2-a299-ee4b0308558a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:12.838820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:12.839909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 306/630] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:12.840729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2a34a-e26f-4dd2-a299-ee4b0308558a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:12.915020 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a10e1-a28b-4c8d-b350-87065e337222 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:12.945177 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2a34a-e26f-4dd2-a299-ee4b0308558a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:12.948483 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8656f-016f-46cf-b615-2581d2f5923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:12.949290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8656f-016f-46cf-b615-2581d2f5923e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.949563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8656f-016f-46cf-b615-2581d2f5923e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.949830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8656f-016f-46cf-b615-2581d2f5923e None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.950458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8656f-016f-46cf-b615-2581d2f5923e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:12.950756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8656f-016f-46cf-b615-2581d2f5923e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:12.952111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8656f-016f-46cf-b615-2581d2f5923e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:12.964647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 325/631] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:12.965864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59745556-5498-40f1-bbb9-c66ceb786c2c None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.966143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59745556-5498-40f1-bbb9-c66ceb786c2c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.966424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59745556-5498-40f1-bbb9-c66ceb786c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.979857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d6da0-c17a-4d0f-9ce0-3a1da0f0eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:12.980492 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d6da0-c17a-4d0f-9ce0-3a1da0f0eee1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:12.980779 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d6da0-c17a-4d0f-9ce0-3a1da0f0eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:12.981052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d6da0-c17a-4d0f-9ce0-3a1da0f0eee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:12.985565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-59745556-5498-40f1-bbb9-c66ceb786c2c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:12.986769 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-59745556-5498-40f1-bbb9-c66ceb786c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:12.991764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d6da0-c17a-4d0f-9ce0-3a1da0f0eee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:12.993237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d6da0-c17a-4d0f-9ce0-3a1da0f0eee1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-855d6da0-c17a-4d0f-9ce0-3a1da0f0eee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:13.007052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:13.008085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:13.009005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:13.009005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:13.009005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:13.009005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 307/632] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:13.009653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3445e-074d-4647-a45b-59d4f285cb82 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:13.009864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3445e-074d-4647-a45b-59d4f285cb82 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:13.010129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3445e-074d-4647-a45b-59d4f285cb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:13.028533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-81f3445e-074d-4647-a45b-59d4f285cb82 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:13.029701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-81f3445e-074d-4647-a45b-59d4f285cb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:13.032958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:13.053824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:13.059260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 326/633] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:13.060069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59745556-5498-40f1-bbb9-c66ceb786c2c None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:13.112395 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 308/634] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:13.113557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3445e-074d-4647-a45b-59d4f285cb82 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:13.177392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59745556-5498-40f1-bbb9-c66ceb786c2c None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:13.219732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a25440-7747-43fd-a7f1-5c8fe7c49ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:13.220840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a25440-7747-43fd-a7f1-5c8fe7c49ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:13.221262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a25440-7747-43fd-a7f1-5c8fe7c49ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:13.221649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a25440-7747-43fd-a7f1-5c8fe7c49ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:13.222647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a25440-7747-43fd-a7f1-5c8fe7c49ea4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:13.223023 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a25440-7747-43fd-a7f1-5c8fe7c49ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:13.224473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a25440-7747-43fd-a7f1-5c8fe7c49ea4 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:13.233715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 327/635] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:13.239812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a25440-7747-43fd-a7f1-5c8fe7c49ea4 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:13.242719 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3445e-074d-4647-a45b-59d4f285cb82 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:13.286124 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e72039-f014-42a1-9821-0af8e5914f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:13.287503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e72039-f014-42a1-9821-0af8e5914f8e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:13.287974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e72039-f014-42a1-9821-0af8e5914f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:13.288804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e72039-f014-42a1-9821-0af8e5914f8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:13.298378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e72039-f014-42a1-9821-0af8e5914f8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:13.300839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e72039-f014-42a1-9821-0af8e5914f8e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:13.304591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:13.308761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-91e72039-f014-42a1-9821-0af8e5914f8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:13.319202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:13.320083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:13.320960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:13.320960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:13.320960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:13.320960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:13.320960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:13.320960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:13.320960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:13.320960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:13.325230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 309/636] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:13.326301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e72039-f014-42a1-9821-0af8e5914f8e None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:13.357280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a25440-7747-43fd-a7f1-5c8fe7c49ea4 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:13.398959 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe5f9d-9cab-4c05-bfa0-1849ba7e584f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:13.399570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe5f9d-9cab-4c05-bfa0-1849ba7e584f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:13.399848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe5f9d-9cab-4c05-bfa0-1849ba7e584f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:13.400122 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe5f9d-9cab-4c05-bfa0-1849ba7e584f None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:13.400821 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe5f9d-9cab-4c05-bfa0-1849ba7e584f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:13.401075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe5f9d-9cab-4c05-bfa0-1849ba7e584f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:13.402714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe5f9d-9cab-4c05-bfa0-1849ba7e584f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:13.415114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 328/637] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:13.415918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe5f9d-9cab-4c05-bfa0-1849ba7e584f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:13.429788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e72039-f014-42a1-9821-0af8e5914f8e None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:13.463358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21ea30-2ecb-4022-a17f-ef6694a129c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:13.463980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21ea30-2ecb-4022-a17f-ef6694a129c2 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:13.464253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21ea30-2ecb-4022-a17f-ef6694a129c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:13.464522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21ea30-2ecb-4022-a17f-ef6694a129c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:13.475460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21ea30-2ecb-4022-a17f-ef6694a129c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:13.476977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21ea30-2ecb-4022-a17f-ef6694a129c2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-cd21ea30-2ecb-4022-a17f-ef6694a129c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:13.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:13.492784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:13.493689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:13.493689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:13.493689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:13.493689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 310/638] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:13.494874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21ea30-2ecb-4022-a17f-ef6694a129c2 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:13.545159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe5f9d-9cab-4c05-bfa0-1849ba7e584f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:13.554182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:13.571324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:13.584932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53266f4c-c058-48f9-b911-816bebeb722a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:13.585571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53266f4c-c058-48f9-b911-816bebeb722a None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:13.585976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53266f4c-c058-48f9-b911-816bebeb722a None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:13.586152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53266f4c-c058-48f9-b911-816bebeb722a None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:13.586824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53266f4c-c058-48f9-b911-816bebeb722a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:13.587131 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53266f4c-c058-48f9-b911-816bebeb722a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:13.588560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53266f4c-c058-48f9-b911-816bebeb722a None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:13.600718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 329/639] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:13.601571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53266f4c-c058-48f9-b911-816bebeb722a None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:13.608669 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21ea30-2ecb-4022-a17f-ef6694a129c2 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:13.642800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9f63b-2a3c-4462-879c-93139de6c2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:13.643421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9f63b-2a3c-4462-879c-93139de6c2dd None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:13.643701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9f63b-2a3c-4462-879c-93139de6c2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:13.643971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9f63b-2a3c-4462-879c-93139de6c2dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:13.654244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9f63b-2a3c-4462-879c-93139de6c2dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-37b9f63b-2a3c-4462-879c-93139de6c2dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:13.659958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:13.661055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 311/640] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:13.662121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9f63b-2a3c-4462-879c-93139de6c2dd None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:13.704519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53266f4c-c058-48f9-b911-816bebeb722a None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:13.738014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff0bb0-d7a9-4305-a147-27b335704afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:13.738638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff0bb0-d7a9-4305-a147-27b335704afc None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:13.738914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff0bb0-d7a9-4305-a147-27b335704afc None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:13.739183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff0bb0-d7a9-4305-a147-27b335704afc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:13.749215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff0bb0-d7a9-4305-a147-27b335704afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-c5ff0bb0-d7a9-4305-a147-27b335704afc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:13.754454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:13.755453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 330/641] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:13.756298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff0bb0-d7a9-4305-a147-27b335704afc None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:13.764038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9f63b-2a3c-4462-879c-93139de6c2dd None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:13.777480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:13.799529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238bb58-24bf-48ba-bfef-802f3f212c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:13.799987 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:13.800123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238bb58-24bf-48ba-bfef-802f3f212c69 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:13.800340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238bb58-24bf-48ba-bfef-802f3f212c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:13.800616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238bb58-24bf-48ba-bfef-802f3f212c69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:13.807884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238bb58-24bf-48ba-bfef-802f3f212c69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:13.809386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238bb58-24bf-48ba-bfef-802f3f212c69 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-d238bb58-24bf-48ba-bfef-802f3f212c69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:13.821404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:13.822460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:13.823406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:13.823406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:13.823406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:13.823406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:13.823406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:13.823406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:13.823406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:13.823406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:13.827276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 312/642] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:13.827964 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238bb58-24bf-48ba-bfef-802f3f212c69 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:13.868097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff0bb0-d7a9-4305-a147-27b335704afc None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:13.901878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86773bc-a1f0-45a0-bb73-314ea4da1dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:13.902480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86773bc-a1f0-45a0-bb73-314ea4da1dfe None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:13.902760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86773bc-a1f0-45a0-bb73-314ea4da1dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:13.903032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86773bc-a1f0-45a0-bb73-314ea4da1dfe None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:13.903666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86773bc-a1f0-45a0-bb73-314ea4da1dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:13.903976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86773bc-a1f0-45a0-bb73-314ea4da1dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:13.905367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86773bc-a1f0-45a0-bb73-314ea4da1dfe None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:13.918245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 331/643] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:13.919053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86773bc-a1f0-45a0-bb73-314ea4da1dfe None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:13.932095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238bb58-24bf-48ba-bfef-802f3f212c69 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:13.966920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b2b48-d5da-466f-b8cb-91461a38d2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:13.967577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b2b48-d5da-466f-b8cb-91461a38d2a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:13.967861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b2b48-d5da-466f-b8cb-91461a38d2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:13.968134 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b2b48-d5da-466f-b8cb-91461a38d2a4 None admin] PATH_INFO: `/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/2f410e0148c1403098b9b73735425726/roles/5354696222554ab2a19a3a839682a597` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:13.982469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b2b48-d5da-466f-b8cb-91461a38d2a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca4d4801f2f40419f6881984fbbfbcd, user_id=2f410e0148c1403098b9b73735425726, role_id=5354696222554ab2a19a3a839682a597)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:13.983907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b2b48-d5da-466f-b8cb-91461a38d2a4 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:13.996509 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 313/644] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:25:13 2022] PUT /identity/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/2f410e0148c1403098b9b73735425726/roles/5354696222554ab2a19a3a839682a597 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:25:13.997360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b2b48-d5da-466f-b8cb-91461a38d2a4 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:14.031363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:14.036356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86773bc-a1f0-45a0-bb73-314ea4da1dfe None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:14.038353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:14.077965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e05f4c-7206-490c-8c11-e3b501c3e48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:14.078536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e05f4c-7206-490c-8c11-e3b501c3e48a None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:14.079318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e05f4c-7206-490c-8c11-e3b501c3e48a None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:14.079601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e05f4c-7206-490c-8c11-e3b501c3e48a None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:14.080290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e05f4c-7206-490c-8c11-e3b501c3e48a None admin] RBAC: Adding query filter params (name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:14.080575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e05f4c-7206-490c-8c11-e3b501c3e48a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:14.082038 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e05f4c-7206-490c-8c11-e3b501c3e48a None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:14.091785 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 332/645] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:14.092670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e05f4c-7206-490c-8c11-e3b501c3e48a None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:14.131076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b2b48-d5da-466f-b8cb-91461a38d2a4 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:14.168767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc6829-8876-4784-99c2-ddaffb7e3119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:14.169423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc6829-8876-4784-99c2-ddaffb7e3119 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:14.169709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc6829-8876-4784-99c2-ddaffb7e3119 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:14.169984 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc6829-8876-4784-99c2-ddaffb7e3119 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:14.170661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc6829-8876-4784-99c2-ddaffb7e3119 None admin] RBAC: Adding query filter params (name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:14.170971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc6829-8876-4784-99c2-ddaffb7e3119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:14.172382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc6829-8876-4784-99c2-ddaffb7e3119 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:14.184821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 314/646] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:14.185716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc6829-8876-4784-99c2-ddaffb7e3119 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:14.212372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e05f4c-7206-490c-8c11-e3b501c3e48a None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:14.249545 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a419985-adba-4b91-89db-9e0dc24272e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:14.250161 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a419985-adba-4b91-89db-9e0dc24272e8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:14.250468 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a419985-adba-4b91-89db-9e0dc24272e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:14.250740 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a419985-adba-4b91-89db-9e0dc24272e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:14.251433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a419985-adba-4b91-89db-9e0dc24272e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:14.251742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a419985-adba-4b91-89db-9e0dc24272e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:14.253188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a419985-adba-4b91-89db-9e0dc24272e8 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:14.265698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 333/647] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:14.266576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a419985-adba-4b91-89db-9e0dc24272e8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:14.287937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:14.296118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:14.307471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc6829-8876-4784-99c2-ddaffb7e3119 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:14.342438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6ca61-c5eb-4529-a578-bf400e2be874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:14.342993 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6ca61-c5eb-4529-a578-bf400e2be874 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:14.343390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6ca61-c5eb-4529-a578-bf400e2be874 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:14.343668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6ca61-c5eb-4529-a578-bf400e2be874 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:14.352078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6ca61-c5eb-4529-a578-bf400e2be874 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:14.353548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6ca61-c5eb-4529-a578-bf400e2be874 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-c6d6ca61-c5eb-4529-a578-bf400e2be874 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:14.364992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:14.365957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:14.366881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:14.366881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:14.366881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:14.366881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:14.366881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:14.366881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:14.366881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:14.366881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:14.366881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 08:25:14.366881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:14.370990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 315/648] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:25:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:14.371748 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6ca61-c5eb-4529-a578-bf400e2be874 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:14.383051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a419985-adba-4b91-89db-9e0dc24272e8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:14.421152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cce3ff-7d7c-44e1-87f4-c77e0882cccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:14.422075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cce3ff-7d7c-44e1-87f4-c77e0882cccd None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:14.422493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cce3ff-7d7c-44e1-87f4-c77e0882cccd None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:14.422902 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cce3ff-7d7c-44e1-87f4-c77e0882cccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:14.434447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cce3ff-7d7c-44e1-87f4-c77e0882cccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:14.436215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cce3ff-7d7c-44e1-87f4-c77e0882cccd None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-c6cce3ff-7d7c-44e1-87f4-c77e0882cccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:14.453100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:14.454738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:14.455814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:14.455814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:14.455814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:14.459595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 334/649] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:14.460387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cce3ff-7d7c-44e1-87f4-c77e0882cccd None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:14.485074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6ca61-c5eb-4529-a578-bf400e2be874 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:14.527982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789299ec-1464-41be-846b-9f0f399ab9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:14.529507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789299ec-1464-41be-846b-9f0f399ab9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:14.529948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789299ec-1464-41be-846b-9f0f399ab9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:14.530228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789299ec-1464-41be-846b-9f0f399ab9f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:14.542567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789299ec-1464-41be-846b-9f0f399ab9f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:14.544017 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789299ec-1464-41be-846b-9f0f399ab9f5 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:14.546822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:14.558858 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-789299ec-1464-41be-846b-9f0f399ab9f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:14.559549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:14.560467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:14.562027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:14.562027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:14.562027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:14.562027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 316/650] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:14.562027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789299ec-1464-41be-846b-9f0f399ab9f5 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:14.594979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cce3ff-7d7c-44e1-87f4-c77e0882cccd None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:14.634916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3512fa-8b03-458a-830b-f53f5a7dadd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:14.635518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3512fa-8b03-458a-830b-f53f5a7dadd7 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:14.635798 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3512fa-8b03-458a-830b-f53f5a7dadd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:14.636085 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3512fa-8b03-458a-830b-f53f5a7dadd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:14.636867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3512fa-8b03-458a-830b-f53f5a7dadd7 None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=9c3c6efc3c154ff89066035cd9583459) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:14.637137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3512fa-8b03-458a-830b-f53f5a7dadd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:14.638591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3512fa-8b03-458a-830b-f53f5a7dadd7 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:14.645152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 335/651] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:14 2022] GET /identity/v3/role_assignments?user.id=9c3c6efc3c154ff89066035cd9583459&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:14.646011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3512fa-8b03-458a-830b-f53f5a7dadd7 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:14.690302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789299ec-1464-41be-846b-9f0f399ab9f5 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:14.742838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52a53f-dc9d-4d5c-a12a-6b005af109a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:14.744329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52a53f-dc9d-4d5c-a12a-6b005af109a9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:14.745259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52a53f-dc9d-4d5c-a12a-6b005af109a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:14.745759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52a53f-dc9d-4d5c-a12a-6b005af109a9 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:14.747055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52a53f-dc9d-4d5c-a12a-6b005af109a9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:14.748191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52a53f-dc9d-4d5c-a12a-6b005af109a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:14.750295 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52a53f-dc9d-4d5c-a12a-6b005af109a9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:14.771494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3512fa-8b03-458a-830b-f53f5a7dadd7 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:14.778237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 317/652] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 08:25:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:14.780276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52a53f-dc9d-4d5c-a12a-6b005af109a9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:14.814042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba798a4-cef4-458f-9bb6-6cf4b14e6169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:14.814754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba798a4-cef4-458f-9bb6-6cf4b14e6169 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:14.815036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba798a4-cef4-458f-9bb6-6cf4b14e6169 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:14.815311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba798a4-cef4-458f-9bb6-6cf4b14e6169 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:14.816054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba798a4-cef4-458f-9bb6-6cf4b14e6169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:14.816416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba798a4-cef4-458f-9bb6-6cf4b14e6169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:14.817859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba798a4-cef4-458f-9bb6-6cf4b14e6169 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:14.829289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:14.833849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 336/653] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:14.834004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba798a4-cef4-458f-9bb6-6cf4b14e6169 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:14.867154 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:14.930664 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52a53f-dc9d-4d5c-a12a-6b005af109a9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:14.972673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba798a4-cef4-458f-9bb6-6cf4b14e6169 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:14.978872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db8567-4baa-460b-9c07-b88e754f9b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:14.979751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db8567-4baa-460b-9c07-b88e754f9b11 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:14.980224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db8567-4baa-460b-9c07-b88e754f9b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:14.980697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db8567-4baa-460b-9c07-b88e754f9b11 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:14.981624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db8567-4baa-460b-9c07-b88e754f9b11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:14.982137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db8567-4baa-460b-9c07-b88e754f9b11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:14.983789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db8567-4baa-460b-9c07-b88e754f9b11 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:14.996577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 318/654] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:14.998425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db8567-4baa-460b-9c07-b88e754f9b11 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:15.011857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74b116-1ec2-4e49-a5e1-5bf48206eb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:15.012477 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74b116-1ec2-4e49-a5e1-5bf48206eb98 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:15.012762 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74b116-1ec2-4e49-a5e1-5bf48206eb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:15.013039 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74b116-1ec2-4e49-a5e1-5bf48206eb98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:15.092576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:15.095506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74b116-1ec2-4e49-a5e1-5bf48206eb98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:15.098206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74b116-1ec2-4e49-a5e1-5bf48206eb98 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:15.119333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db8567-4baa-460b-9c07-b88e754f9b11 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:15.136081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:15.159979 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de01460-2ccd-46d2-8073-735f0fa793f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:15.161037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de01460-2ccd-46d2-8073-735f0fa793f3 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:15.161488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de01460-2ccd-46d2-8073-735f0fa793f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:15.161940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de01460-2ccd-46d2-8073-735f0fa793f3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:15.170487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de01460-2ccd-46d2-8073-735f0fa793f3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:15.172224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de01460-2ccd-46d2-8073-735f0fa793f3 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-1de01460-2ccd-46d2-8073-735f0fa793f3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:15.184740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:15.185912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:15.186973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:15.186973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:15.186973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:15.186973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:15.186973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:15.186973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:15.186973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:15.186973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:15.186973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 08:25:15.186973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:15.189138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 319/655] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:15.190346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de01460-2ccd-46d2-8073-735f0fa793f3 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:15.221274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 337/656] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:25:15.222533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74b116-1ec2-4e49-a5e1-5bf48206eb98 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:15.317195 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de01460-2ccd-46d2-8073-735f0fa793f3 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:15.357053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:15.359142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b55de-28a1-4c2e-8e4e-576cd9354bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:15.359737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b55de-28a1-4c2e-8e4e-576cd9354bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:15.361462 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b55de-28a1-4c2e-8e4e-576cd9354bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:15.361741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74b116-1ec2-4e49-a5e1-5bf48206eb98 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:15.362468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b55de-28a1-4c2e-8e4e-576cd9354bc8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:15.373178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b55de-28a1-4c2e-8e4e-576cd9354bc8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:15.375717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b55de-28a1-4c2e-8e4e-576cd9354bc8 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-d10b55de-28a1-4c2e-8e4e-576cd9354bc8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:15.392475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:15.393699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:15.394835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:15.394835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:15.394835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:15.394835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:15.394835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:15.394835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:15.394835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:15.394835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:15.394835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 08:25:15.394835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:15.401903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 320/657] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:25:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:15.402755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b55de-28a1-4c2e-8e4e-576cd9354bc8 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:15.411681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df99084-e162-4674-8ede-1f7d9817338b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:15.412589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df99084-e162-4674-8ede-1f7d9817338b None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:15.413618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df99084-e162-4674-8ede-1f7d9817338b None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:15.414106 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df99084-e162-4674-8ede-1f7d9817338b None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:15.415511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df99084-e162-4674-8ede-1f7d9817338b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:15.416076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df99084-e162-4674-8ede-1f7d9817338b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:15.418667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df99084-e162-4674-8ede-1f7d9817338b None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:15.443918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 338/658] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Sep 28 08:25:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:15.444249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:15.444792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df99084-e162-4674-8ede-1f7d9817338b None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:15.546968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b55de-28a1-4c2e-8e4e-576cd9354bc8 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:15.560962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df99084-e162-4674-8ede-1f7d9817338b None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:15.593504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219e8af-d89c-49a0-91e9-32bbf3ea9ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:15.595263 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219e8af-d89c-49a0-91e9-32bbf3ea9ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:15.595548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219e8af-d89c-49a0-91e9-32bbf3ea9ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:15.595821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219e8af-d89c-49a0-91e9-32bbf3ea9ac1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:15.600794 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dd2a8-9f67-45ee-b7c3-09ba803e9e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:15.602031 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dd2a8-9f67-45ee-b7c3-09ba803e9e01 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:15.602517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dd2a8-9f67-45ee-b7c3-09ba803e9e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:15.603564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dd2a8-9f67-45ee-b7c3-09ba803e9e01 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:15.604515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dd2a8-9f67-45ee-b7c3-09ba803e9e01 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:15.605543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dd2a8-9f67-45ee-b7c3-09ba803e9e01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:15.607277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219e8af-d89c-49a0-91e9-32bbf3ea9ac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:15.607693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dd2a8-9f67-45ee-b7c3-09ba803e9e01 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:15.608711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219e8af-d89c-49a0-91e9-32bbf3ea9ac1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-a219e8af-d89c-49a0-91e9-32bbf3ea9ac1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:15.624738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:15.625768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:15.626671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:15.626671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:15.626671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:15.630451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 321/659] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:15.631367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219e8af-d89c-49a0-91e9-32bbf3ea9ac1 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:15.636914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 339/660] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 08:25:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:15.638019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dd2a8-9f67-45ee-b7c3-09ba803e9e01 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:15.657622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:15.749560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:15.751308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dd2a8-9f67-45ee-b7c3-09ba803e9e01 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:15.767816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219e8af-d89c-49a0-91e9-32bbf3ea9ac1 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:15.799681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c296a-ddc8-4fc1-aa56-a7ee01459170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:15.800293 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c296a-ddc8-4fc1-aa56-a7ee01459170 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:15.800570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c296a-ddc8-4fc1-aa56-a7ee01459170 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:15.800839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c296a-ddc8-4fc1-aa56-a7ee01459170 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:15.801511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c296a-ddc8-4fc1-aa56-a7ee01459170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:15.801811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c296a-ddc8-4fc1-aa56-a7ee01459170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:15.803224 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c296a-ddc8-4fc1-aa56-a7ee01459170 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:15.808681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a533b4-a9bd-4707-8fcf-57982449b6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:15.809631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a533b4-a9bd-4707-8fcf-57982449b6cf None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:15.810668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a533b4-a9bd-4707-8fcf-57982449b6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:15.811188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a533b4-a9bd-4707-8fcf-57982449b6cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:15.820112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 340/661] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:15.821029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c296a-ddc8-4fc1-aa56-a7ee01459170 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:15.899540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a533b4-a9bd-4707-8fcf-57982449b6cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:15.902062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a533b4-a9bd-4707-8fcf-57982449b6cf None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:15.902711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:15.936795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c296a-ddc8-4fc1-aa56-a7ee01459170 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:15.986329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acca04-809f-4b44-9d86-d42f372afc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:15.987144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acca04-809f-4b44-9d86-d42f372afc56 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:15.987543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acca04-809f-4b44-9d86-d42f372afc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:15.987920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acca04-809f-4b44-9d86-d42f372afc56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:16.000284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:16.019063 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 322/662] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:25:16.020348 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a533b4-a9bd-4707-8fcf-57982449b6cf None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:16.082982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acca04-809f-4b44-9d86-d42f372afc56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:16.084906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23acca04-809f-4b44-9d86-d42f372afc56 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:16.170052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a533b4-a9bd-4707-8fcf-57982449b6cf None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:16.183871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:16.213590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 341/663] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:25:16.214089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254a1f9-94d3-49ce-870c-4d4c22f0e666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:16.214489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acca04-809f-4b44-9d86-d42f372afc56 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:16.215213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254a1f9-94d3-49ce-870c-4d4c22f0e666 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:16.215701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254a1f9-94d3-49ce-870c-4d4c22f0e666 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:16.216183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254a1f9-94d3-49ce-870c-4d4c22f0e666 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:16.224346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254a1f9-94d3-49ce-870c-4d4c22f0e666 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:16.226091 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254a1f9-94d3-49ce-870c-4d4c22f0e666 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-7254a1f9-94d3-49ce-870c-4d4c22f0e666 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:16.239609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:16.240504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:16.241636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:16.241636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:16.241636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:16.241636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:16.241636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:16.241636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:16.241636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:16.241636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:16.242363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 323/664] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:16.242363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd29993-2491-4bef-b9db-1bb904fa53f8 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:16.242363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd29993-2491-4bef-b9db-1bb904fa53f8 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:16.242363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd29993-2491-4bef-b9db-1bb904fa53f8 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:16.242363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 324/665] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:16.242363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd29993-2491-4bef-b9db-1bb904fa53f8 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:16.296475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:16.331209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acca04-809f-4b44-9d86-d42f372afc56 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:16.355029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd29993-2491-4bef-b9db-1bb904fa53f8 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:16.369039 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d82030-51a6-4c26-8105-b274bd11e325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:16.369705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d82030-51a6-4c26-8105-b274bd11e325 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:16.369982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d82030-51a6-4c26-8105-b274bd11e325 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:16.370254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d82030-51a6-4c26-8105-b274bd11e325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:16.381353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d82030-51a6-4c26-8105-b274bd11e325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:16.382918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d82030-51a6-4c26-8105-b274bd11e325 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:16.391590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a6442-aa69-45a5-a41e-653053ab25bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:16.392136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a6442-aa69-45a5-a41e-653053ab25bf None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:16.392411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a6442-aa69-45a5-a41e-653053ab25bf None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:16.392687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a6442-aa69-45a5-a41e-653053ab25bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-c3d82030-51a6-4c26-8105-b274bd11e325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:16.398971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:16.399884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:16.405802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:16.405802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:16.405802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:16.405802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 342/666] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:16.405802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d82030-51a6-4c26-8105-b274bd11e325 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:16.411153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a6442-aa69-45a5-a41e-653053ab25bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:16.412685 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a6442-aa69-45a5-a41e-653053ab25bf None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-004a6442-aa69-45a5-a41e-653053ab25bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:16.428674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:16.429695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:16.430578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:16.430578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:16.430578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:16.430578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 325/667] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:16.430879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8d36a-936c-4275-87c8-8ed4be5bf8f5 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:16.431160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8d36a-936c-4275-87c8-8ed4be5bf8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:16.431437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8d36a-936c-4275-87c8-8ed4be5bf8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:16.441385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:16.450634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-5ab8d36a-936c-4275-87c8-8ed4be5bf8f5 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:16.451818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5ab8d36a-936c-4275-87c8-8ed4be5bf8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:16.519674 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d82030-51a6-4c26-8105-b274bd11e325 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:16.529235 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 326/668] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:16.530410 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8d36a-936c-4275-87c8-8ed4be5bf8f5 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:16.561002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d10d8-3004-4aaa-9855-b115483b627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:16.561654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d10d8-3004-4aaa-9855-b115483b627f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:16.561931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d10d8-3004-4aaa-9855-b115483b627f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:16.562206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d10d8-3004-4aaa-9855-b115483b627f None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:16.562982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d10d8-3004-4aaa-9855-b115483b627f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:16.563307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d10d8-3004-4aaa-9855-b115483b627f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:16.564723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d10d8-3004-4aaa-9855-b115483b627f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:16.581126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 343/669] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:16.581126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d10d8-3004-4aaa-9855-b115483b627f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:16.582539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:16.678624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8d36a-936c-4275-87c8-8ed4be5bf8f5 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:16.683147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:16.699691 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d10d8-3004-4aaa-9855-b115483b627f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:16.723909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18b4fc-76ac-404a-98bf-4db84c6e5cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:16.724392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18b4fc-76ac-404a-98bf-4db84c6e5cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:16.724666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18b4fc-76ac-404a-98bf-4db84c6e5cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:16.724936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18b4fc-76ac-404a-98bf-4db84c6e5cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:16.725610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18b4fc-76ac-404a-98bf-4db84c6e5cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:16.725927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18b4fc-76ac-404a-98bf-4db84c6e5cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:16.727332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18b4fc-76ac-404a-98bf-4db84c6e5cd5 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:16.739533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 327/670] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:16.740380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18b4fc-76ac-404a-98bf-4db84c6e5cd5 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:16.752588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dab546-ad7b-4cac-bc6c-ec22e003245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:16.753994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dab546-ad7b-4cac-bc6c-ec22e003245f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:16.754765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dab546-ad7b-4cac-bc6c-ec22e003245f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:16.755242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dab546-ad7b-4cac-bc6c-ec22e003245f None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:16.756356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dab546-ad7b-4cac-bc6c-ec22e003245f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:16.757380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dab546-ad7b-4cac-bc6c-ec22e003245f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:16.759268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dab546-ad7b-4cac-bc6c-ec22e003245f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:16.776872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 344/671] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:16.778960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dab546-ad7b-4cac-bc6c-ec22e003245f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:16.837160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:16.850262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18b4fc-76ac-404a-98bf-4db84c6e5cd5 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:16.891161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41d608-d639-4317-9b0d-68b5347901f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:16.891915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41d608-d639-4317-9b0d-68b5347901f8 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:16.892103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41d608-d639-4317-9b0d-68b5347901f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:16.892352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41d608-d639-4317-9b0d-68b5347901f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:16.893073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41d608-d639-4317-9b0d-68b5347901f8 None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=5274740e5d7c4ab1ba9e9781815532c9) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:16.893427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41d608-d639-4317-9b0d-68b5347901f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:16.894923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41d608-d639-4317-9b0d-68b5347901f8 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:16.902255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 328/672] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:16 2022] GET /identity/v3/role_assignments?user.id=5274740e5d7c4ab1ba9e9781815532c9&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:16.902983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41d608-d639-4317-9b0d-68b5347901f8 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:16.927665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dab546-ad7b-4cac-bc6c-ec22e003245f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:16.982272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a34650-dd90-478e-b750-224e3ed752e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:16.983296 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a34650-dd90-478e-b750-224e3ed752e1 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:16.983583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a34650-dd90-478e-b750-224e3ed752e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:16.983862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a34650-dd90-478e-b750-224e3ed752e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:16.994848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a34650-dd90-478e-b750-224e3ed752e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:16.996347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a34650-dd90-478e-b750-224e3ed752e1 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-91a34650-dd90-478e-b750-224e3ed752e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:17.010105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:17.011123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:17.012005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:17.012005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:17.012005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:17.012005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:17.012005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:17.012005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:17.012005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:17.012005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:17.012005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 345/673] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:17.012492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a34650-dd90-478e-b750-224e3ed752e1 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:17.029065 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:17.029296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41d608-d639-4317-9b0d-68b5347901f8 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:17.069100 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971dd71-112e-4a81-bf9e-6f7daa7fb1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:17.069771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971dd71-112e-4a81-bf9e-6f7daa7fb1ab None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:17.070052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971dd71-112e-4a81-bf9e-6f7daa7fb1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:17.070324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971dd71-112e-4a81-bf9e-6f7daa7fb1ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:17.077890 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971dd71-112e-4a81-bf9e-6f7daa7fb1ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:17.079406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971dd71-112e-4a81-bf9e-6f7daa7fb1ab None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-a971dd71-112e-4a81-bf9e-6f7daa7fb1ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:17.091820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:17.092756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:17.093688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:17.093688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:17.093688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:17.093688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:17.093688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:17.093688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:17.093688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:17.093688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:17.093688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 329/674] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:17.094980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386fb07-dbaa-4049-8e43-bfafdb2389e9 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:17.094980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386fb07-dbaa-4049-8e43-bfafdb2389e9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:17.095544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386fb07-dbaa-4049-8e43-bfafdb2389e9 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:17.096901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 330/675] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:17.098708 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386fb07-dbaa-4049-8e43-bfafdb2389e9 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:17.118442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:17.143937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a34650-dd90-478e-b750-224e3ed752e1 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:17.185571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ec816-12d8-4495-beb4-992657cca774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:17.187367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ec816-12d8-4495-beb4-992657cca774 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:17.187367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ec816-12d8-4495-beb4-992657cca774 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:17.187367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ec816-12d8-4495-beb4-992657cca774 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:17.200980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ec816-12d8-4495-beb4-992657cca774 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-8b8ec816-12d8-4495-beb4-992657cca774 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:17.206360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:17.207340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:17.208176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 346/676] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:17.208875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ec816-12d8-4495-beb4-992657cca774 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:17.226495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386fb07-dbaa-4049-8e43-bfafdb2389e9 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:17.262108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a52cd-4b59-402d-b93a-07a2b6904b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:17.262634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a52cd-4b59-402d-b93a-07a2b6904b28 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:17.262910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a52cd-4b59-402d-b93a-07a2b6904b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:17.263180 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a52cd-4b59-402d-b93a-07a2b6904b28 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:17.263907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a52cd-4b59-402d-b93a-07a2b6904b28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:17.264213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a52cd-4b59-402d-b93a-07a2b6904b28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:17.265681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a52cd-4b59-402d-b93a-07a2b6904b28 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:17.280254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:17.284689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 331/677] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:17.286153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5037a6-632c-4ef0-bc31-005670446cd5 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:17.286434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5037a6-632c-4ef0-bc31-005670446cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:17.286701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5037a6-632c-4ef0-bc31-005670446cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:17.304683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-7a5037a6-632c-4ef0-bc31-005670446cd5 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:17.305889 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7a5037a6-632c-4ef0-bc31-005670446cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:17.320760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ec816-12d8-4495-beb4-992657cca774 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:17.360578 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b71dd-f458-428f-b40d-7745b8e9d61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:17.361299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b71dd-f458-428f-b40d-7745b8e9d61e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:17.361639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b71dd-f458-428f-b40d-7745b8e9d61e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:17.361958 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b71dd-f458-428f-b40d-7745b8e9d61e None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:17.362665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b71dd-f458-428f-b40d-7745b8e9d61e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:17.362962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b71dd-f458-428f-b40d-7745b8e9d61e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:17.364392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b71dd-f458-428f-b40d-7745b8e9d61e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:17.376752 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 347/678] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:17.377605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b71dd-f458-428f-b40d-7745b8e9d61e None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:17.384024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 332/679] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:17.385221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5037a6-632c-4ef0-bc31-005670446cd5 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:17.390568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:17.501153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5037a6-632c-4ef0-bc31-005670446cd5 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:17.516634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b71dd-f458-428f-b40d-7745b8e9d61e None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:17.543320 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d944e2-2974-45c4-8237-53b8e3535d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:17.543931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d944e2-2974-45c4-8237-53b8e3535d72 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:17.544212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d944e2-2974-45c4-8237-53b8e3535d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:17.544487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d944e2-2974-45c4-8237-53b8e3535d72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:17.545264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d944e2-2974-45c4-8237-53b8e3535d72 None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=736efafe9fc94b45bc75566b1696c498) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:17.545999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d944e2-2974-45c4-8237-53b8e3535d72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:17.547785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d944e2-2974-45c4-8237-53b8e3535d72 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:17.553542 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:17.561117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 333/680] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:17 2022] GET /identity/v3/role_assignments?user.id=736efafe9fc94b45bc75566b1696c498&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:17.561117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d944e2-2974-45c4-8237-53b8e3535d72 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:17.577688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc1e69-7606-4b15-89d6-00428e5ad6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:17.578919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc1e69-7606-4b15-89d6-00428e5ad6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:17.579203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc1e69-7606-4b15-89d6-00428e5ad6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:17.579951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc1e69-7606-4b15-89d6-00428e5ad6e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:17.580475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc1e69-7606-4b15-89d6-00428e5ad6e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:17.580783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc1e69-7606-4b15-89d6-00428e5ad6e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:17.582212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc1e69-7606-4b15-89d6-00428e5ad6e3 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:17.606179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 348/681] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:17.612009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc1e69-7606-4b15-89d6-00428e5ad6e3 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:17.697173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:17.752549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d944e2-2974-45c4-8237-53b8e3535d72 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:17.762913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc1e69-7606-4b15-89d6-00428e5ad6e3 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:17.794066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7263ad2-6d71-473b-bd8a-e19cbe646d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:17.794601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7263ad2-6d71-473b-bd8a-e19cbe646d26 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:17.794876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7263ad2-6d71-473b-bd8a-e19cbe646d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:17.795153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7263ad2-6d71-473b-bd8a-e19cbe646d26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:17.795875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7263ad2-6d71-473b-bd8a-e19cbe646d26 None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=ff22c2632c614fca8008feab93c8fa2d) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:17.796182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7263ad2-6d71-473b-bd8a-e19cbe646d26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:17.797714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7263ad2-6d71-473b-bd8a-e19cbe646d26 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:17.805455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 334/682] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:17 2022] GET /identity/v3/role_assignments?user.id=ff22c2632c614fca8008feab93c8fa2d&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:17.806297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7263ad2-6d71-473b-bd8a-e19cbe646d26 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:17.814064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016eaef0-983f-4bdd-b5e9-f3862c3140d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:17.814813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eaef0-983f-4bdd-b5e9-f3862c3140d5 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:17.815097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eaef0-983f-4bdd-b5e9-f3862c3140d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:17.815372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eaef0-983f-4bdd-b5e9-f3862c3140d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:17.829401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016eaef0-983f-4bdd-b5e9-f3862c3140d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:17.830932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016eaef0-983f-4bdd-b5e9-f3862c3140d5 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-016eaef0-983f-4bdd-b5e9-f3862c3140d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:17.849855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:17.850870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:17.851839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:17.851839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:17.851839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:17.851839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 349/683] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:17.857706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016eaef0-983f-4bdd-b5e9-f3862c3140d5 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:17.912978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:17.979346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7263ad2-6d71-473b-bd8a-e19cbe646d26 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:17.992920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016eaef0-983f-4bdd-b5e9-f3862c3140d5 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:18.034869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b55301-4739-4148-9dbd-3915cdb107fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:18.035506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b55301-4739-4148-9dbd-3915cdb107fa None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:18.035789 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b55301-4739-4148-9dbd-3915cdb107fa None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:18.036067 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b55301-4739-4148-9dbd-3915cdb107fa None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:18.036804 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b55301-4739-4148-9dbd-3915cdb107fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:18.037141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b55301-4739-4148-9dbd-3915cdb107fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:18.039065 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b55301-4739-4148-9dbd-3915cdb107fa None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:18.042436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:18.057033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0b4b1-380e-406e-a5bc-23a85e8b48cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:18.059140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0b4b1-380e-406e-a5bc-23a85e8b48cc None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:18.059140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0b4b1-380e-406e-a5bc-23a85e8b48cc None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:18.059140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0b4b1-380e-406e-a5bc-23a85e8b48cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:18.063417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 350/684] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:18.064934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b55301-4739-4148-9dbd-3915cdb107fa None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:18.086687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0b4b1-380e-406e-a5bc-23a85e8b48cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-bce0b4b1-380e-406e-a5bc-23a85e8b48cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:18.095683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:18.096659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:18.098514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 335/685] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:25:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:18.104145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0b4b1-380e-406e-a5bc-23a85e8b48cc None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:18.191826 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b55301-4739-4148-9dbd-3915cdb107fa None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:18.194480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:18.243234 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea632923-f5f6-48cf-826b-11d0f4aa2071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:18.243611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0b4b1-380e-406e-a5bc-23a85e8b48cc None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:18.243872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea632923-f5f6-48cf-826b-11d0f4aa2071 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:18.244151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea632923-f5f6-48cf-826b-11d0f4aa2071 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:18.244422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea632923-f5f6-48cf-826b-11d0f4aa2071 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:18.253216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea632923-f5f6-48cf-826b-11d0f4aa2071 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:18.254780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea632923-f5f6-48cf-826b-11d0f4aa2071 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-ea632923-f5f6-48cf-826b-11d0f4aa2071 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:18.271916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:18.272911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:18.273866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:18.273866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:18.273866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:18.273866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:18.273866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:18.273866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:18.273866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:18.273866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:18.273866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 08:25:18.273866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:18.273866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 351/686] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:25:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:18.286128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea632923-f5f6-48cf-826b-11d0f4aa2071 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:18.292317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cac52a-5ad9-41f8-8a6e-52af0d5a14fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:18.293002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cac52a-5ad9-41f8-8a6e-52af0d5a14fb None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:18.293324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cac52a-5ad9-41f8-8a6e-52af0d5a14fb None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:18.293598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cac52a-5ad9-41f8-8a6e-52af0d5a14fb None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:18.294324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cac52a-5ad9-41f8-8a6e-52af0d5a14fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:18.294632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cac52a-5ad9-41f8-8a6e-52af0d5a14fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:18.296003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cac52a-5ad9-41f8-8a6e-52af0d5a14fb None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:18.307420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 336/687] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:25:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:18.314231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cac52a-5ad9-41f8-8a6e-52af0d5a14fb None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:18.375197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:18.427076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea632923-f5f6-48cf-826b-11d0f4aa2071 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:18.433790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cac52a-5ad9-41f8-8a6e-52af0d5a14fb None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:18.465214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be612e-a914-4d14-b9a1-36a60d3f4659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:18.466428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be612e-a914-4d14-b9a1-36a60d3f4659 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:18.466982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be612e-a914-4d14-b9a1-36a60d3f4659 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:18.467515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be612e-a914-4d14-b9a1-36a60d3f4659 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:18.468669 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be612e-a914-4d14-b9a1-36a60d3f4659 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:18.469318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be612e-a914-4d14-b9a1-36a60d3f4659 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:18.471063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be612e-a914-4d14-b9a1-36a60d3f4659 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:18.488478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b2a95-a8fd-43dc-b066-9fa1b3a2afe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:18.489163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2a95-a8fd-43dc-b066-9fa1b3a2afe1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:18.489451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2a95-a8fd-43dc-b066-9fa1b3a2afe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:18.489724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2a95-a8fd-43dc-b066-9fa1b3a2afe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:18.495639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 352/688] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 08:25:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:18.504819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be612e-a914-4d14-b9a1-36a60d3f4659 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:18.506241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b2a95-a8fd-43dc-b066-9fa1b3a2afe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:18.508160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b2a95-a8fd-43dc-b066-9fa1b3a2afe1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:18.536228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-c49b2a95-a8fd-43dc-b066-9fa1b3a2afe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:18.553486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:18.554717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:18.555816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:18.555816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:18.555816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:18.555816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 337/689] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:18.559976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b2a95-a8fd-43dc-b066-9fa1b3a2afe1 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:18.708056 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be612e-a914-4d14-b9a1-36a60d3f4659 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:18.744475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b2a95-a8fd-43dc-b066-9fa1b3a2afe1 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:18.749945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:18.771268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4232b-eaab-4fa2-a18d-f098237513ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:18.772565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4232b-eaab-4fa2-a18d-f098237513ba None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:18.773056 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4232b-eaab-4fa2-a18d-f098237513ba None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:18.773642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4232b-eaab-4fa2-a18d-f098237513ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:18.826867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c28f1-f4c3-4b01-a639-879ad54ae374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:18.827415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c28f1-f4c3-4b01-a639-879ad54ae374 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:18.827698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c28f1-f4c3-4b01-a639-879ad54ae374 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:18.827971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c28f1-f4c3-4b01-a639-879ad54ae374 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:18.829210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c28f1-f4c3-4b01-a639-879ad54ae374 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:18.829516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c28f1-f4c3-4b01-a639-879ad54ae374 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:18.830888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c28f1-f4c3-4b01-a639-879ad54ae374 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:18.851652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 338/690] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:18.856740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c28f1-f4c3-4b01-a639-879ad54ae374 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:18.887211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4232b-eaab-4fa2-a18d-f098237513ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:18.888682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4232b-eaab-4fa2-a18d-f098237513ba None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:18.917103 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:19.023384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 353/691] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:25:19.023752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c28f1-f4c3-4b01-a639-879ad54ae374 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:19.029337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4232b-eaab-4fa2-a18d-f098237513ba None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:19.094889 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a2cdb-6e97-4642-b96c-d5f381625ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:19.095538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a2cdb-6e97-4642-b96c-d5f381625ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:19.095733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a2cdb-6e97-4642-b96c-d5f381625ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:19.096007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a2cdb-6e97-4642-b96c-d5f381625ec1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:19.114357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a2cdb-6e97-4642-b96c-d5f381625ec1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:19.115815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a2cdb-6e97-4642-b96c-d5f381625ec1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-308a2cdb-6e97-4642-b96c-d5f381625ec1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:19.133441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:19.134518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:19.135435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:19.135435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:19.135435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:19.135435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:19.135435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:19.135435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:19.135435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:19.135435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:19.135435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:25:19.135435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:19.144696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 339/692] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 08:25:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:19.145665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a2cdb-6e97-4642-b96c-d5f381625ec1 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:19.178421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4232b-eaab-4fa2-a18d-f098237513ba None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:19.201633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:19.217585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:19.277798 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab25618-ff4a-4038-9491-4ffda4d4a907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:19.278453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab25618-ff4a-4038-9491-4ffda4d4a907 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:19.278735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab25618-ff4a-4038-9491-4ffda4d4a907 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:19.279012 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab25618-ff4a-4038-9491-4ffda4d4a907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:19.299631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a2cdb-6e97-4642-b96c-d5f381625ec1 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:19.303127 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab25618-ff4a-4038-9491-4ffda4d4a907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:19.304583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab25618-ff4a-4038-9491-4ffda4d4a907 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:19.338498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec6ca2-9a25-4aaf-8139-c7d2815473d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-4ab25618-ff4a-4038-9491-4ffda4d4a907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:19.338818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:19.339694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:19.340579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:19.340579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:19.340579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:19.340746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec6ca2-9a25-4aaf-8139-c7d2815473d8 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:19.340746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec6ca2-9a25-4aaf-8139-c7d2815473d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:19.340746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec6ca2-9a25-4aaf-8139-c7d2815473d8 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:19.340942 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 354/693] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:19.341011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec6ca2-9a25-4aaf-8139-c7d2815473d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:19.341011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec6ca2-9a25-4aaf-8139-c7d2815473d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:19.347077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec6ca2-9a25-4aaf-8139-c7d2815473d8 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:19.350043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab25618-ff4a-4038-9491-4ffda4d4a907 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:19.373053 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 340/694] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 08:25:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:19.374680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08100d55-daf2-4782-b152-5c076007b298 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:19.374972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08100d55-daf2-4782-b152-5c076007b298 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:19.375873 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08100d55-daf2-4782-b152-5c076007b298 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:19.377631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 341/695] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:19.382280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08100d55-daf2-4782-b152-5c076007b298 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:19.478046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab25618-ff4a-4038-9491-4ffda4d4a907 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:19.493400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:19.517884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957545c6-6590-4b20-b037-0b04c9d89b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:19.518777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957545c6-6590-4b20-b037-0b04c9d89b8b None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:19.518894 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08100d55-daf2-4782-b152-5c076007b298 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:19.519298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957545c6-6590-4b20-b037-0b04c9d89b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:19.519725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957545c6-6590-4b20-b037-0b04c9d89b8b None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:19.520508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957545c6-6590-4b20-b037-0b04c9d89b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:19.520965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957545c6-6590-4b20-b037-0b04c9d89b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:19.522494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957545c6-6590-4b20-b037-0b04c9d89b8b None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:19.541857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 355/696] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:19.543470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cabb43-7906-46a4-8e93-5f91588fbaaa None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:19.543860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cabb43-7906-46a4-8e93-5f91588fbaaa None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:19.544235 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cabb43-7906-46a4-8e93-5f91588fbaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:19.564246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89648596-a41e-4b7b-bc4b-73b84ec99dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:19.565108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-57cabb43-7906-46a4-8e93-5f91588fbaaa None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:19.565287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89648596-a41e-4b7b-bc4b-73b84ec99dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:19.565729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89648596-a41e-4b7b-bc4b-73b84ec99dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:19.566146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89648596-a41e-4b7b-bc4b-73b84ec99dd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:19.566807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-57cabb43-7906-46a4-8e93-5f91588fbaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:19.598569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:19.653503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 356/697] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:19.654778 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cabb43-7906-46a4-8e93-5f91588fbaaa None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:19.655566 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89648596-a41e-4b7b-bc4b-73b84ec99dd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:19.657841 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89648596-a41e-4b7b-bc4b-73b84ec99dd7 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:19.753166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:19.791529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 342/698] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:25:19.801146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89648596-a41e-4b7b-bc4b-73b84ec99dd7 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:19.858699 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cabb43-7906-46a4-8e93-5f91588fbaaa None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:19.929622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc457b58-ab17-4502-af25-8c7da7be8429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:19.930274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc457b58-ab17-4502-af25-8c7da7be8429 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:19.930556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc457b58-ab17-4502-af25-8c7da7be8429 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:19.930832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc457b58-ab17-4502-af25-8c7da7be8429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:19.944589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc457b58-ab17-4502-af25-8c7da7be8429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:19.946080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc457b58-ab17-4502-af25-8c7da7be8429 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-bc457b58-ab17-4502-af25-8c7da7be8429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:19.974009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:19.974950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:19.975823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:19.975823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:19.975823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:19.975823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:19.975823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:19.975823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:19.975823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:19.975823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:19.976703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 357/699] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:19.983294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc457b58-ab17-4502-af25-8c7da7be8429 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:19.999744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89648596-a41e-4b7b-bc4b-73b84ec99dd7 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:20.027291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:20.051310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdf7c7-bdbc-42df-8610-1371c10250e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:20.051720 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdf7c7-bdbc-42df-8610-1371c10250e8 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:20.052001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdf7c7-bdbc-42df-8610-1371c10250e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:20.052272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdf7c7-bdbc-42df-8610-1371c10250e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:20.064478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdf7c7-bdbc-42df-8610-1371c10250e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:20.066127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdf7c7-bdbc-42df-8610-1371c10250e8 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-c9fdf7c7-bdbc-42df-8610-1371c10250e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:20.083558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:20.084684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:20.085626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:20.085626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:20.085626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:20.085626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 343/700] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:20.086332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdf7c7-bdbc-42df-8610-1371c10250e8 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:20.127373 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc457b58-ab17-4502-af25-8c7da7be8429 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:20.127550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:20.169590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f07d07-203f-4824-be22-8d90d66947bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:20.170462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f07d07-203f-4824-be22-8d90d66947bf None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:20.170919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f07d07-203f-4824-be22-8d90d66947bf None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:20.171324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f07d07-203f-4824-be22-8d90d66947bf None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:20.172185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f07d07-203f-4824-be22-8d90d66947bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:20.172640 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f07d07-203f-4824-be22-8d90d66947bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:20.174122 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f07d07-203f-4824-be22-8d90d66947bf None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:20.188908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 358/701] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:20.189829 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f07d07-203f-4824-be22-8d90d66947bf None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:20.217906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdf7c7-bdbc-42df-8610-1371c10250e8 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:20.263800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed9fc7-d393-4546-b8ae-210f39f998ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:20.264460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed9fc7-d393-4546-b8ae-210f39f998ed None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:20.264739 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed9fc7-d393-4546-b8ae-210f39f998ed None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:20.265020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed9fc7-d393-4546-b8ae-210f39f998ed None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:20.265637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed9fc7-d393-4546-b8ae-210f39f998ed None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:20.267581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed9fc7-d393-4546-b8ae-210f39f998ed None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-e8ed9fc7-d393-4546-b8ae-210f39f998ed None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:20.279274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:20.280346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:20.286953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 344/702] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Sep 28 08:25:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:20.287766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01629ffa-76ec-4430-bf52-83abfa7aa206 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:20.287766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01629ffa-76ec-4430-bf52-83abfa7aa206 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:20.287766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01629ffa-76ec-4430-bf52-83abfa7aa206 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:20.287766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 345/703] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:20.287766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01629ffa-76ec-4430-bf52-83abfa7aa206 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:20.296511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:20.319759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f07d07-203f-4824-be22-8d90d66947bf None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:20.358521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164161c-5660-4323-8980-fe246f729f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:20.359141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164161c-5660-4323-8980-fe246f729f42 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:20.359422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164161c-5660-4323-8980-fe246f729f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:20.360414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164161c-5660-4323-8980-fe246f729f42 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:20.360907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164161c-5660-4323-8980-fe246f729f42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:20.361246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164161c-5660-4323-8980-fe246f729f42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:20.363199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164161c-5660-4323-8980-fe246f729f42 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:20.375861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 359/704] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:20.376725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164161c-5660-4323-8980-fe246f729f42 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:20.404788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:20.410983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01629ffa-76ec-4430-bf52-83abfa7aa206 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:20.457530 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681153e2-3055-41f6-a2d8-1049d68b61d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:20.458153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681153e2-3055-41f6-a2d8-1049d68b61d2 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:20.458437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681153e2-3055-41f6-a2d8-1049d68b61d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:20.458715 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681153e2-3055-41f6-a2d8-1049d68b61d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:20.459449 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681153e2-3055-41f6-a2d8-1049d68b61d2 None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=2f410e0148c1403098b9b73735425726) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:20.459771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681153e2-3055-41f6-a2d8-1049d68b61d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:20.461290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681153e2-3055-41f6-a2d8-1049d68b61d2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:20.467379 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 346/705] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:20 2022] GET /identity/v3/role_assignments?user.id=2f410e0148c1403098b9b73735425726&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:20.468591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c97e8-55e7-45f7-a600-a76162714e07 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:20.468940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c97e8-55e7-45f7-a600-a76162714e07 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:20.469738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c97e8-55e7-45f7-a600-a76162714e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:20.489806 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-af2c97e8-55e7-45f7-a600-a76162714e07 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:20.491060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-af2c97e8-55e7-45f7-a600-a76162714e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:20.530217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164161c-5660-4323-8980-fe246f729f42 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:20.592711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3b291-b6e4-4ad9-8301-3b48b12da8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:20.594725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3b291-b6e4-4ad9-8301-3b48b12da8fa None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:20.595231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3b291-b6e4-4ad9-8301-3b48b12da8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:20.595707 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3b291-b6e4-4ad9-8301-3b48b12da8fa None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:20.596840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3b291-b6e4-4ad9-8301-3b48b12da8fa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:20.597437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3b291-b6e4-4ad9-8301-3b48b12da8fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:20.600140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3b291-b6e4-4ad9-8301-3b48b12da8fa None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:20.614970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 360/706] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 08:25:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:20.614970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3b291-b6e4-4ad9-8301-3b48b12da8fa None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:20.618056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 347/707] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:20.619267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c97e8-55e7-45f7-a600-a76162714e07 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:20.632042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:20.753616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:20.766133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c97e8-55e7-45f7-a600-a76162714e07 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:20.816003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640c3b7-f834-45a2-ad90-4442ecc8fc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:20.817952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640c3b7-f834-45a2-ad90-4442ecc8fc8d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:20.818467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640c3b7-f834-45a2-ad90-4442ecc8fc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:20.819137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640c3b7-f834-45a2-ad90-4442ecc8fc8d None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:20.820205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640c3b7-f834-45a2-ad90-4442ecc8fc8d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:20.820767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640c3b7-f834-45a2-ad90-4442ecc8fc8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:20.822832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640c3b7-f834-45a2-ad90-4442ecc8fc8d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:20.830901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 348/708] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 08:25:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:20.845133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706edb09-c9ec-4098-bd9a-b05c34dd7afa None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:20.845133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706edb09-c9ec-4098-bd9a-b05c34dd7afa None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:20.845432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706edb09-c9ec-4098-bd9a-b05c34dd7afa None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:20.846760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 349/709] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:20.856041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf82b5-333a-44ca-bfbc-b061c470f1c6 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:20.856303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf82b5-333a-44ca-bfbc-b061c470f1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:20.856651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf82b5-333a-44ca-bfbc-b061c470f1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:20.863923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3b291-b6e4-4ad9-8301-3b48b12da8fa None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:20.885861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-3ddf82b5-333a-44ca-bfbc-b061c470f1c6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:20.887122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3ddf82b5-333a-44ca-bfbc-b061c470f1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:20.948569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9f3b1-27a4-4d5a-85c1-d60441e7b980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:20.950955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9f3b1-27a4-4d5a-85c1-d60441e7b980 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:20.951963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9f3b1-27a4-4d5a-85c1-d60441e7b980 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:20.952743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9f3b1-27a4-4d5a-85c1-d60441e7b980 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:20.965041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9f3b1-27a4-4d5a-85c1-d60441e7b980 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:20.966651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9f3b1-27a4-4d5a-85c1-d60441e7b980 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-23c9f3b1-27a4-4d5a-85c1-d60441e7b980 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:20.984757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:20.985976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:20.986891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:20.986891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:20.986891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:20.986891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:20.986891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:20.986891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:20.986891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:20.986891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:20.986891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 361/710] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:20.987871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9f3b1-27a4-4d5a-85c1-d60441e7b980 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:20.998513 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 350/711] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 08:25:21.001938 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf82b5-333a-44ca-bfbc-b061c470f1c6 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:21.079661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:21.144521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:21.150646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9f3b1-27a4-4d5a-85c1-d60441e7b980 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:21.168117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf82b5-333a-44ca-bfbc-b061c470f1c6 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:21.201031 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8316f-4049-4fdf-beb0-a60b2177cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:21.201727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8316f-4049-4fdf-beb0-a60b2177cb45 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:21.202003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8316f-4049-4fdf-beb0-a60b2177cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:21.202285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8316f-4049-4fdf-beb0-a60b2177cb45 None admin] PATH_INFO: `/v3/services/network` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:21.202862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8316f-4049-4fdf-beb0-a60b2177cb45 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:21.204806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8316f-4049-4fdf-beb0-a60b2177cb45 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-84e8316f-4049-4fdf-beb0-a60b2177cb45 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:21.226147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:21.228139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 08:25:21.229879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:21.239163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 362/712] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:20 2022] GET /identity/v3/services/network => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:21.248003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8316f-4049-4fdf-beb0-a60b2177cb45 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:21.253756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb1053-d62c-4777-8052-a88b3d553621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:21.255563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb1053-d62c-4777-8052-a88b3d553621 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:21.256547 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb1053-d62c-4777-8052-a88b3d553621 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:21.257448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb1053-d62c-4777-8052-a88b3d553621 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:21.259153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb1053-d62c-4777-8052-a88b3d553621 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:21.260005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb1053-d62c-4777-8052-a88b3d553621 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:21.263329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb1053-d62c-4777-8052-a88b3d553621 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:21.289662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 351/713] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:21.291911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb1053-d62c-4777-8052-a88b3d553621 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:21.411517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8316f-4049-4fdf-beb0-a60b2177cb45 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:21.457547 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb1053-d62c-4777-8052-a88b3d553621 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:21.460010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:21.470639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68074cae-a10d-477b-b4d5-ccd69d6483e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:21.472079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68074cae-a10d-477b-b4d5-ccd69d6483e2 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:21.472560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68074cae-a10d-477b-b4d5-ccd69d6483e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:21.473436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68074cae-a10d-477b-b4d5-ccd69d6483e2 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:21.474132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68074cae-a10d-477b-b4d5-ccd69d6483e2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:21.474446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68074cae-a10d-477b-b4d5-ccd69d6483e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:21.475854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68074cae-a10d-477b-b4d5-ccd69d6483e2 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:21.485179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 363/714] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:25:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:21.485179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68074cae-a10d-477b-b4d5-ccd69d6483e2 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:21.508824 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0467b-665e-4e10-937a-4c5cbd2a165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:21.509911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0467b-665e-4e10-937a-4c5cbd2a165b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:21.510588 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0467b-665e-4e10-937a-4c5cbd2a165b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:21.511313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0467b-665e-4e10-937a-4c5cbd2a165b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:21.540179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0467b-665e-4e10-937a-4c5cbd2a165b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:21.542007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0467b-665e-4e10-937a-4c5cbd2a165b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:21.550452 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-6cd0467b-665e-4e10-937a-4c5cbd2a165b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:21.559807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:21.560798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:21.562388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:21.562388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:21.562388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:21.562388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 352/715] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:21.562388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0467b-665e-4e10-937a-4c5cbd2a165b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:21.627568 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68074cae-a10d-477b-b4d5-ccd69d6483e2 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:21.671391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bf2dd-5233-4f1f-8549-e4e6bf3c8572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:21.672904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bf2dd-5233-4f1f-8549-e4e6bf3c8572 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:21.673424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bf2dd-5233-4f1f-8549-e4e6bf3c8572 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:21.673891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bf2dd-5233-4f1f-8549-e4e6bf3c8572 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:21.675346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bf2dd-5233-4f1f-8549-e4e6bf3c8572 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=9c3c6efc3c154ff89066035cd9583459) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:21.675951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bf2dd-5233-4f1f-8549-e4e6bf3c8572 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:21.679078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bf2dd-5233-4f1f-8549-e4e6bf3c8572 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:21.689533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 364/716] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:21 2022] GET /identity/v3/role_assignments?user.id=9c3c6efc3c154ff89066035cd9583459&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:21.690693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bf2dd-5233-4f1f-8549-e4e6bf3c8572 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:21.700814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0467b-665e-4e10-937a-4c5cbd2a165b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:21.758628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d568e-27cd-4d32-95f3-1e0e206a69ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:21.759349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d568e-27cd-4d32-95f3-1e0e206a69ac None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:21.759636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d568e-27cd-4d32-95f3-1e0e206a69ac None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:21.759912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d568e-27cd-4d32-95f3-1e0e206a69ac None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:21.762719 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d568e-27cd-4d32-95f3-1e0e206a69ac None admin] RBAC: Adding query filter params (type=network) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:21.763062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d568e-27cd-4d32-95f3-1e0e206a69ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:21.764452 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d568e-27cd-4d32-95f3-1e0e206a69ac None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:21.785125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 353/717] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:25:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:21.815289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:21.878253 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bf2dd-5233-4f1f-8549-e4e6bf3c8572 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:21.932912 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b193272-7259-4bea-b6e4-66d81b3a4dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:21.933962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b193272-7259-4bea-b6e4-66d81b3a4dbf None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:21.934440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b193272-7259-4bea-b6e4-66d81b3a4dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:21.934911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b193272-7259-4bea-b6e4-66d81b3a4dbf None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:21.935833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b193272-7259-4bea-b6e4-66d81b3a4dbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:21.936342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b193272-7259-4bea-b6e4-66d81b3a4dbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:21.938058 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b193272-7259-4bea-b6e4-66d81b3a4dbf None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:21.959345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 365/718] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:21.965023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d568e-27cd-4d32-95f3-1e0e206a69ac None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:22.073311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:22.129853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d568e-27cd-4d32-95f3-1e0e206a69ac None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:22.197910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb2d66-bed3-49fd-8385-c7cbbbf9d536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:22.198860 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb2d66-bed3-49fd-8385-c7cbbbf9d536 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:22.199348 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb2d66-bed3-49fd-8385-c7cbbbf9d536 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:22.199708 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb2d66-bed3-49fd-8385-c7cbbbf9d536 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:22.200599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb2d66-bed3-49fd-8385-c7cbbbf9d536 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:22.202675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb2d66-bed3-49fd-8385-c7cbbbf9d536 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:22.243331 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 354/719] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:25:21 2022] POST /identity/v3/users => generated 254 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:25:23.298484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f7fee-866c-4f8b-88e0-0e8208efac92 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:23.299019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f7fee-866c-4f8b-88e0-0e8208efac92 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:23.299019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f7fee-866c-4f8b-88e0-0e8208efac92 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:23.300593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 366/720] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:23.306394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d5b94-a505-49d5-8af5-6c33d66ccb3e None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:23.306871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d5b94-a505-49d5-8af5-6c33d66ccb3e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:23.306996 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d5b94-a505-49d5-8af5-6c33d66ccb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:23.326869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-c53d5b94-a505-49d5-8af5-6c33d66ccb3e None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:23.328094 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c53d5b94-a505-49d5-8af5-6c33d66ccb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:23.414270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 355/721] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:23.635142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4ba3e-bc3d-406f-bc19-4bfab6324f88 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:23.635445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4ba3e-bc3d-406f-bc19-4bfab6324f88 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:23.635631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4ba3e-bc3d-406f-bc19-4bfab6324f88 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:23.637340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 367/722] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:23.644046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa0c4c-e439-43a4-adb7-f30dda1e1324 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:23.644482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa0c4c-e439-43a4-adb7-f30dda1e1324 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:23.644784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa0c4c-e439-43a4-adb7-f30dda1e1324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:23.674160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-61fa0c4c-e439-43a4-adb7-f30dda1e1324 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:23.675374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-61fa0c4c-e439-43a4-adb7-f30dda1e1324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:23.731172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:23.763144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4ba3e-bc3d-406f-bc19-4bfab6324f88 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:23.767883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 356/723] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:23.906591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4ba3e-bc3d-406f-bc19-4bfab6324f88 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:23.927091 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:23.946744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd35e0-d395-4140-b7e8-8b2fee7c996b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:23.947405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd35e0-d395-4140-b7e8-8b2fee7c996b None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:23.947683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd35e0-d395-4140-b7e8-8b2fee7c996b None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:23.947960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd35e0-d395-4140-b7e8-8b2fee7c996b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:23.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd35e0-d395-4140-b7e8-8b2fee7c996b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-f4fd35e0-d395-4140-b7e8-8b2fee7c996b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:23.969948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:23.971030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:23.972261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 368/724] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:25:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:23.977352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa0c4c-e439-43a4-adb7-f30dda1e1324 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:24.118194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa0c4c-e439-43a4-adb7-f30dda1e1324 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:24.134587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd35e0-d395-4140-b7e8-8b2fee7c996b None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:24.161982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4da98-ca54-45a8-bf21-242195450303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:24.162527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4da98-ca54-45a8-bf21-242195450303 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:24.162809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4da98-ca54-45a8-bf21-242195450303 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:24.163180 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4da98-ca54-45a8-bf21-242195450303 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:24.164021 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4da98-ca54-45a8-bf21-242195450303 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:24.164327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4da98-ca54-45a8-bf21-242195450303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:24.165781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4da98-ca54-45a8-bf21-242195450303 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:24.182768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 357/725] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:25:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:24.200360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4da98-ca54-45a8-bf21-242195450303 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:24.255972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd35e0-d395-4140-b7e8-8b2fee7c996b None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:24.268978 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:24.338196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4da98-ca54-45a8-bf21-242195450303 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:24.346057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c2a74-bb07-4130-8f7e-26a157249bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:24.346709 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c2a74-bb07-4130-8f7e-26a157249bfa None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:24.346949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c2a74-bb07-4130-8f7e-26a157249bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:24.347223 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c2a74-bb07-4130-8f7e-26a157249bfa None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:24.347895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c2a74-bb07-4130-8f7e-26a157249bfa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:24.349286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c2a74-bb07-4130-8f7e-26a157249bfa None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:24.380600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 369/726] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:25:24 2022] POST /identity/v3/services => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:25:24.407234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b5ebd-4053-4ade-b679-7baf43c89368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:24.407898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b5ebd-4053-4ade-b679-7baf43c89368 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:24.408174 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b5ebd-4053-4ade-b679-7baf43c89368 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:24.408436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b5ebd-4053-4ade-b679-7baf43c89368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:24.427093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b5ebd-4053-4ade-b679-7baf43c89368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:24.428535 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b5ebd-4053-4ade-b679-7baf43c89368 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-775b5ebd-4053-4ade-b679-7baf43c89368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:24.449414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:24.450403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:24.451875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:24.451875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:24.451875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:24.451875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 358/727] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:24.456613 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c2a74-bb07-4130-8f7e-26a157249bfa None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:24.536179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:24.630289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c2a74-bb07-4130-8f7e-26a157249bfa None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:24.672588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb2476-3148-4c18-bcbf-6f6ca1facffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:24.673313 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb2476-3148-4c18-bcbf-6f6ca1facffc None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:24.673594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb2476-3148-4c18-bcbf-6f6ca1facffc None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:24.673865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb2476-3148-4c18-bcbf-6f6ca1facffc None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:24.674544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb2476-3148-4c18-bcbf-6f6ca1facffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:24.674847 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb2476-3148-4c18-bcbf-6f6ca1facffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:24.676289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb2476-3148-4c18-bcbf-6f6ca1facffc None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:24.690306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 370/728] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:24.698483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b5ebd-4053-4ade-b679-7baf43c89368 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:24.709256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d48fa1-da46-4945-b9f5-d7e5e3cb94a1 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:24.709487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d48fa1-da46-4945-b9f5-d7e5e3cb94a1 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:24.709763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d48fa1-da46-4945-b9f5-d7e5e3cb94a1 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:24.713800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 371/729] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:24.717712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461fa9-b817-40f5-b458-903a22cff9d9 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:24.719581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461fa9-b817-40f5-b458-903a22cff9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:24.720035 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461fa9-b817-40f5-b458-903a22cff9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:24.747903 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-43461fa9-b817-40f5-b458-903a22cff9d9 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:24.749438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-43461fa9-b817-40f5-b458-903a22cff9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:24.831813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 372/730] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:24.833154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c079ce3-8979-4859-b71c-22fc54d58685 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:24.833440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c079ce3-8979-4859-b71c-22fc54d58685 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:24.833717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c079ce3-8979-4859-b71c-22fc54d58685 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:24.834389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:24.835268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 373/731] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:24.841001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e17288-2e3d-4a24-b202-a04b032e8a6f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:24.841510 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e17288-2e3d-4a24-b202-a04b032e8a6f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:24.841824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e17288-2e3d-4a24-b202-a04b032e8a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:24.851603 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b5ebd-4053-4ade-b679-7baf43c89368 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:24.882190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-94e17288-2e3d-4a24-b202-a04b032e8a6f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:24.883930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-94e17288-2e3d-4a24-b202-a04b032e8a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:24.896712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762b9bc-6116-4e24-ba3a-6eecb309a306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:24.897715 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762b9bc-6116-4e24-ba3a-6eecb309a306 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:24.898209 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762b9bc-6116-4e24-ba3a-6eecb309a306 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:24.898664 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762b9bc-6116-4e24-ba3a-6eecb309a306 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:24.909307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762b9bc-6116-4e24-ba3a-6eecb309a306 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:24.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762b9bc-6116-4e24-ba3a-6eecb309a306 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-5762b9bc-6116-4e24-ba3a-6eecb309a306 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:24.925727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:24.926737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:24.927684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:24.927684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:24.927684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:24.927684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:24.927684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:24.927684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:24.927684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:24.927684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:24.927684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 08:25:24.927684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:24.930454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 359/732] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:25:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:24.937027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762b9bc-6116-4e24-ba3a-6eecb309a306 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:24.942473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:25.061581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 374/733] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 08:25:25.083450 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762b9bc-6116-4e24-ba3a-6eecb309a306 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:25.120721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6e8f6-0d2a-4371-93c0-04bb3fedfd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:25.121667 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e8f6-0d2a-4371-93c0-04bb3fedfd02 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:25.122099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e8f6-0d2a-4371-93c0-04bb3fedfd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:25.122542 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e8f6-0d2a-4371-93c0-04bb3fedfd02 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:25.123441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6e8f6-0d2a-4371-93c0-04bb3fedfd02 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:25.123895 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6e8f6-0d2a-4371-93c0-04bb3fedfd02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:25.126156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6e8f6-0d2a-4371-93c0-04bb3fedfd02 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:25.152887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 360/734] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 08:25:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:25.159183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e17288-2e3d-4a24-b202-a04b032e8a6f None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:25.253544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6e8f6-0d2a-4371-93c0-04bb3fedfd02 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:25.273503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:25.301620 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e17288-2e3d-4a24-b202-a04b032e8a6f None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:25.368697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03bcd9-2fc5-46db-a558-0b880f862e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:25.371934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03bcd9-2fc5-46db-a558-0b880f862e10 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:25.372449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03bcd9-2fc5-46db-a558-0b880f862e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:25.372996 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03bcd9-2fc5-46db-a558-0b880f862e10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:25.431632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:25.467828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6e8f6-0d2a-4371-93c0-04bb3fedfd02 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:25.504071 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03bcd9-2fc5-46db-a558-0b880f862e10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:25.505906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03bcd9-2fc5-46db-a558-0b880f862e10 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:25.509626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c829c6-24e7-446b-8fc8-36ec64c6ca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:25.510012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c829c6-24e7-446b-8fc8-36ec64c6ca33 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:25.510358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c829c6-24e7-446b-8fc8-36ec64c6ca33 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:25.510696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c829c6-24e7-446b-8fc8-36ec64c6ca33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:25.532762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c829c6-24e7-446b-8fc8-36ec64c6ca33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-66c829c6-24e7-446b-8fc8-36ec64c6ca33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:25.538215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:25.539401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:25.540252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:25.540252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:25.540252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:25.540252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:25.540252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:25.540252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:25.540252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:25.540252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:25.540252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:25.540252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:25.540252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:25:25.540252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:25.546332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 361/735] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:25:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:25.546332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c829c6-24e7-446b-8fc8-36ec64c6ca33 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:25.637909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:25.653956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 375/736] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:25 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 08:25:25.655417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03bcd9-2fc5-46db-a558-0b880f862e10 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:25.709575 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c829c6-24e7-446b-8fc8-36ec64c6ca33 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:25.727961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:25.756788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257e202-8e9a-4b3b-881e-1cc1658395fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:25.757336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257e202-8e9a-4b3b-881e-1cc1658395fb None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:25.757617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257e202-8e9a-4b3b-881e-1cc1658395fb None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:25.757887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257e202-8e9a-4b3b-881e-1cc1658395fb None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:25.758541 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257e202-8e9a-4b3b-881e-1cc1658395fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:25.759904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257e202-8e9a-4b3b-881e-1cc1658395fb None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:25.778381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03bcd9-2fc5-46db-a558-0b880f862e10 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:25.782151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 362/737] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:25:25 2022] POST /identity/v3/services => generated 240 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:25:25.785946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257e202-8e9a-4b3b-881e-1cc1658395fb None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:25.840313 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983f8f4-aa7d-4f9c-8448-e5e52ec85863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:25.841573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983f8f4-aa7d-4f9c-8448-e5e52ec85863 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:25.842203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983f8f4-aa7d-4f9c-8448-e5e52ec85863 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:25.842689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983f8f4-aa7d-4f9c-8448-e5e52ec85863 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:25.846090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983f8f4-aa7d-4f9c-8448-e5e52ec85863 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:25.846090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983f8f4-aa7d-4f9c-8448-e5e52ec85863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:25.847133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983f8f4-aa7d-4f9c-8448-e5e52ec85863 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:25.863461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 376/738] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:25:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:25.866417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288e0cb-f7e1-4f39-b861-84eaf8db8c03 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:25.867292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288e0cb-f7e1-4f39-b861-84eaf8db8c03 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:25.868091 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288e0cb-f7e1-4f39-b861-84eaf8db8c03 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:25.877424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 377/739] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:25.877424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288e0cb-f7e1-4f39-b861-84eaf8db8c03 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:25.960853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257e202-8e9a-4b3b-881e-1cc1658395fb None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:26.006216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:26.022673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380430f-7878-41bf-b17f-6641ba094cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:26.023234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380430f-7878-41bf-b17f-6641ba094cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:26.023520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380430f-7878-41bf-b17f-6641ba094cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:26.023820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380430f-7878-41bf-b17f-6641ba094cd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:26.035367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288e0cb-f7e1-4f39-b861-84eaf8db8c03 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:26.037184 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380430f-7878-41bf-b17f-6641ba094cd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:26.038658 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380430f-7878-41bf-b17f-6641ba094cd4 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-d380430f-7878-41bf-b17f-6641ba094cd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:26.062423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:26.063848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:26.065832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:26.065832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:26.065832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:26.065832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 363/740] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:26.067411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:26.070545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b3a47-470d-44c4-809a-3d794474ca5a None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:26.070545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b3a47-470d-44c4-809a-3d794474ca5a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:26.070545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b3a47-470d-44c4-809a-3d794474ca5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:26.090139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53f954-f953-4dda-8beb-1b9d5e4d9681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:26.090775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53f954-f953-4dda-8beb-1b9d5e4d9681 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:26.091052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53f954-f953-4dda-8beb-1b9d5e4d9681 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:26.091325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53f954-f953-4dda-8beb-1b9d5e4d9681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:26.093596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-6e0b3a47-470d-44c4-809a-3d794474ca5a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:26.095415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6e0b3a47-470d-44c4-809a-3d794474ca5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:26.104639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53f954-f953-4dda-8beb-1b9d5e4d9681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:26.106201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53f954-f953-4dda-8beb-1b9d5e4d9681 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-ff53f954-f953-4dda-8beb-1b9d5e4d9681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:26.125045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:26.126060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:26.126938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:26.126938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:26.126938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:26.128562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 378/741] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:26.128562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53f954-f953-4dda-8beb-1b9d5e4d9681 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:26.190608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 364/742] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:26.191820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b3a47-470d-44c4-809a-3d794474ca5a None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:26.253234 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53f954-f953-4dda-8beb-1b9d5e4d9681 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:26.295892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432ccda-e751-4009-a9a8-47317152e0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:26.297118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432ccda-e751-4009-a9a8-47317152e0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:26.297659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432ccda-e751-4009-a9a8-47317152e0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:26.298139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432ccda-e751-4009-a9a8-47317152e0e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:26.299105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2432ccda-e751-4009-a9a8-47317152e0e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:26.299723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2432ccda-e751-4009-a9a8-47317152e0e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:26.301357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2432ccda-e751-4009-a9a8-47317152e0e8 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:26.316673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 379/743] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:26.330242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432ccda-e751-4009-a9a8-47317152e0e8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:26.347183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b3a47-470d-44c4-809a-3d794474ca5a None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:26.363912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:26.372108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:26.419054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50c9e6-c6b5-4878-947d-110ab8d8cfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:26.422701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50c9e6-c6b5-4878-947d-110ab8d8cfd0 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:26.422998 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50c9e6-c6b5-4878-947d-110ab8d8cfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:26.423280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50c9e6-c6b5-4878-947d-110ab8d8cfd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:26.423957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50c9e6-c6b5-4878-947d-110ab8d8cfd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:26.424265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50c9e6-c6b5-4878-947d-110ab8d8cfd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:26.425722 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50c9e6-c6b5-4878-947d-110ab8d8cfd0 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:26.446985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 365/744] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:26.452541 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50c9e6-c6b5-4878-947d-110ab8d8cfd0 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:26.533445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432ccda-e751-4009-a9a8-47317152e0e8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:26.592706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a67768-c796-404d-a20e-e56184ccb9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:26.593308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a67768-c796-404d-a20e-e56184ccb9dd None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:26.593588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a67768-c796-404d-a20e-e56184ccb9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:26.594724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a67768-c796-404d-a20e-e56184ccb9dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:26.604854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a67768-c796-404d-a20e-e56184ccb9dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:26.606618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a67768-c796-404d-a20e-e56184ccb9dd None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:26.614687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50c9e6-c6b5-4878-947d-110ab8d8cfd0 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-76a67768-c796-404d-a20e-e56184ccb9dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:26.621954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:26.622917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:26.624279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:26.624279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:26.624279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:26.624279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:26.624279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:26.624279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:26.624279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:26.624279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:26.624279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 380/745] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:26.624279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a67768-c796-404d-a20e-e56184ccb9dd None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:26.684065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7802e9-2a54-4efd-86c4-7451dacc09c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:26.685231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7802e9-2a54-4efd-86c4-7451dacc09c3 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:26.685733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7802e9-2a54-4efd-86c4-7451dacc09c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:26.686224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7802e9-2a54-4efd-86c4-7451dacc09c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:26.697783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7802e9-2a54-4efd-86c4-7451dacc09c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:26.699556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7802e9-2a54-4efd-86c4-7451dacc09c3 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-1d7802e9-2a54-4efd-86c4-7451dacc09c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:26.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:26.724445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:26.726346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:26.726346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:26.726346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:26.726346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:26.726346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:26.726346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:26.726346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:26.726346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:26.726346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:25:26.726346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:26.730779 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 366/746] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 08:25:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:26.732145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7802e9-2a54-4efd-86c4-7451dacc09c3 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:26.777662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:26.785349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:26.813251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a67768-c796-404d-a20e-e56184ccb9dd None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:26.867372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6f611-1e67-42e6-bd1c-246c373ca97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:26.868742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6f611-1e67-42e6-bd1c-246c373ca97d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:26.869859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6f611-1e67-42e6-bd1c-246c373ca97d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:26.870366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6f611-1e67-42e6-bd1c-246c373ca97d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:26.889658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6f611-1e67-42e6-bd1c-246c373ca97d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:26.891462 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7802e9-2a54-4efd-86c4-7451dacc09c3 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-45e6f611-1e67-42e6-bd1c-246c373ca97d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:26.896860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:26.898029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 381/747] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:26.899375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6f611-1e67-42e6-bd1c-246c373ca97d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:26.935555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d7dd-17ea-491a-ad26-281d9c50519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:26.936628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d7dd-17ea-491a-ad26-281d9c50519c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:26.937139 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d7dd-17ea-491a-ad26-281d9c50519c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:26.938151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d7dd-17ea-491a-ad26-281d9c50519c None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:26.939055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d7dd-17ea-491a-ad26-281d9c50519c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:26.939590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d7dd-17ea-491a-ad26-281d9c50519c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:26.941294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d7dd-17ea-491a-ad26-281d9c50519c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:26.956809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 367/748] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:26.958270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d7dd-17ea-491a-ad26-281d9c50519c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:27.056454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6f611-1e67-42e6-bd1c-246c373ca97d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:27.090888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:27.108105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109c1ef-013d-4975-9b8f-36511430ff72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:27.109022 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109c1ef-013d-4975-9b8f-36511430ff72 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:27.109523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109c1ef-013d-4975-9b8f-36511430ff72 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:27.109797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109c1ef-013d-4975-9b8f-36511430ff72 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:27.109975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d7dd-17ea-491a-ad26-281d9c50519c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:27.110801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109c1ef-013d-4975-9b8f-36511430ff72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:27.111105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109c1ef-013d-4975-9b8f-36511430ff72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:27.113839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109c1ef-013d-4975-9b8f-36511430ff72 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:27.136347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 382/749] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 08:25:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:27.137228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109c1ef-013d-4975-9b8f-36511430ff72 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:27.151852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:27.159753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8fc09-c45b-4bf7-9243-1ac5282fdaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:27.160378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8fc09-c45b-4bf7-9243-1ac5282fdaaf None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:27.160662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8fc09-c45b-4bf7-9243-1ac5282fdaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:27.161200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8fc09-c45b-4bf7-9243-1ac5282fdaaf None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:27.161649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8fc09-c45b-4bf7-9243-1ac5282fdaaf None admin] RBAC: Adding query filter params (name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:27.162398 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8fc09-c45b-4bf7-9243-1ac5282fdaaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:27.163558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8fc09-c45b-4bf7-9243-1ac5282fdaaf None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:27.179242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 368/750] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:27.179242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a29844-6505-4ae4-84dc-0fbc8fa4cf33 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:27.179242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a29844-6505-4ae4-84dc-0fbc8fa4cf33 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:27.179242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a29844-6505-4ae4-84dc-0fbc8fa4cf33 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:27.179242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 369/751] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:27.179242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a29844-6505-4ae4-84dc-0fbc8fa4cf33 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:27.314865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a29844-6505-4ae4-84dc-0fbc8fa4cf33 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:27.316580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109c1ef-013d-4975-9b8f-36511430ff72 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:27.369605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9790ea-ab70-4188-b8a6-17ff6c2ad505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:27.370675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9790ea-ab70-4188-b8a6-17ff6c2ad505 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:27.371178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9790ea-ab70-4188-b8a6-17ff6c2ad505 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:27.371680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9790ea-ab70-4188-b8a6-17ff6c2ad505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:27.372597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52325514-8166-4364-b974-53fa0cff39de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:27.373125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9790ea-ab70-4188-b8a6-17ff6c2ad505 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=2f410e0148c1403098b9b73735425726) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:27.373555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52325514-8166-4364-b974-53fa0cff39de None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:27.373555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52325514-8166-4364-b974-53fa0cff39de None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:27.373708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9790ea-ab70-4188-b8a6-17ff6c2ad505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:27.373780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52325514-8166-4364-b974-53fa0cff39de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:27.376533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9790ea-ab70-4188-b8a6-17ff6c2ad505 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:27.386748 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52325514-8166-4364-b974-53fa0cff39de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:27.397280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52325514-8166-4364-b974-53fa0cff39de None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:27.397522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 383/752] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:27 2022] GET /identity/v3/role_assignments?user.id=2f410e0148c1403098b9b73735425726&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:27.397522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dcff5-7f97-42dd-a28e-db190ea6e79f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:27.397522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dcff5-7f97-42dd-a28e-db190ea6e79f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:27.397522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dcff5-7f97-42dd-a28e-db190ea6e79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-52325514-8166-4364-b974-53fa0cff39de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:27.417678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:27.418718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:27.419600 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:27.419600 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:27.419600 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:27.423859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 370/753] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:27.424918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52325514-8166-4364-b974-53fa0cff39de None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:27.428320 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:27.429800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-349dcff5-7f97-42dd-a28e-db190ea6e79f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:27.431459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-349dcff5-7f97-42dd-a28e-db190ea6e79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:27.518053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:27.530171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 384/754] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:27.530171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dcff5-7f97-42dd-a28e-db190ea6e79f None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:27.667221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52325514-8166-4364-b974-53fa0cff39de None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:27.780481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27caf9bf-1093-44a4-b157-9bde3ccdadd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:27.781948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caf9bf-1093-44a4-b157-9bde3ccdadd5 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:27.782363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caf9bf-1093-44a4-b157-9bde3ccdadd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:27.782765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27caf9bf-1093-44a4-b157-9bde3ccdadd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:27.791355 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dcff5-7f97-42dd-a28e-db190ea6e79f None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:27.815439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27caf9bf-1093-44a4-b157-9bde3ccdadd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:27.816931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27caf9bf-1093-44a4-b157-9bde3ccdadd5 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-27caf9bf-1093-44a4-b157-9bde3ccdadd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:27.839515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:27.840521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:27.841477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:27.841477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:27.841477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:27.841477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 371/755] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:27.847243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27caf9bf-1093-44a4-b157-9bde3ccdadd5 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:27.873920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131aa9f-1508-49e3-9fd2-2fa4bf66b0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:27.875386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131aa9f-1508-49e3-9fd2-2fa4bf66b0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:27.875898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131aa9f-1508-49e3-9fd2-2fa4bf66b0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:27.876367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131aa9f-1508-49e3-9fd2-2fa4bf66b0d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:27.877283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131aa9f-1508-49e3-9fd2-2fa4bf66b0d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:27.878537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131aa9f-1508-49e3-9fd2-2fa4bf66b0d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:27.880500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131aa9f-1508-49e3-9fd2-2fa4bf66b0d2 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:27.905184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 385/756] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:27.907134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131aa9f-1508-49e3-9fd2-2fa4bf66b0d2 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:27.936042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:27.994901 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:28.023607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27caf9bf-1093-44a4-b157-9bde3ccdadd5 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:28.058325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131aa9f-1508-49e3-9fd2-2fa4bf66b0d2 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:28.078886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fb26a-f2b9-4b39-8e30-c1c59cdc9293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:28.078886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fb26a-f2b9-4b39-8e30-c1c59cdc9293 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:28.078886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fb26a-f2b9-4b39-8e30-c1c59cdc9293 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:28.078886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fb26a-f2b9-4b39-8e30-c1c59cdc9293 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:28.079308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fb26a-f2b9-4b39-8e30-c1c59cdc9293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:28.079308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fb26a-f2b9-4b39-8e30-c1c59cdc9293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:28.079308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fb26a-f2b9-4b39-8e30-c1c59cdc9293 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:28.102980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 372/757] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:28.104725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fb26a-f2b9-4b39-8e30-c1c59cdc9293 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:28.131683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b47f7c-3991-4579-92f7-ab814a85bf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:28.132695 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b47f7c-3991-4579-92f7-ab814a85bf8f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:28.133307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b47f7c-3991-4579-92f7-ab814a85bf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:28.133781 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b47f7c-3991-4579-92f7-ab814a85bf8f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:28.134902 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b47f7c-3991-4579-92f7-ab814a85bf8f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:28.137144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b47f7c-3991-4579-92f7-ab814a85bf8f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-47b47f7c-3991-4579-92f7-ab814a85bf8f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:28.153854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:28.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 08:25:28.155712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:28.164591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 386/758] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Sep 28 08:25:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:28.166580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b47f7c-3991-4579-92f7-ab814a85bf8f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:28.261659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fb26a-f2b9-4b39-8e30-c1c59cdc9293 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:28.278836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:28.307635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6af6d-0937-452c-971b-1e0aa77398e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:28.307904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6af6d-0937-452c-971b-1e0aa77398e3 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:28.308073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6af6d-0937-452c-971b-1e0aa77398e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:28.308410 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6af6d-0937-452c-971b-1e0aa77398e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:28.316293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6af6d-0937-452c-971b-1e0aa77398e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:28.318140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6af6d-0937-452c-971b-1e0aa77398e3 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-dfb6af6d-0937-452c-971b-1e0aa77398e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:28.330412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:28.331365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:28.332232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:28.332232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:28.332232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:28.332232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:28.332232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:28.332232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:28.332232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:28.332232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:28.335534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b47f7c-3991-4579-92f7-ab814a85bf8f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:28.336597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 373/759] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:28.337595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6af6d-0937-452c-971b-1e0aa77398e3 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:28.368906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:28.384311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49448b6b-7df8-408e-a76d-1c51ba1d1e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:28.385404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49448b6b-7df8-408e-a76d-1c51ba1d1e1e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:28.385793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49448b6b-7df8-408e-a76d-1c51ba1d1e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:28.386210 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49448b6b-7df8-408e-a76d-1c51ba1d1e1e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:28.394877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49448b6b-7df8-408e-a76d-1c51ba1d1e1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:28.396344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49448b6b-7df8-408e-a76d-1c51ba1d1e1e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-49448b6b-7df8-408e-a76d-1c51ba1d1e1e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:28.412399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:28.413633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:28.414551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:28.414551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:28.414551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:28.414551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:28.414551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:28.414551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:28.414551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:28.414551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:28.414551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 08:25:28.414551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:28.414551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 387/760] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 08:25:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:28.414551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49448b6b-7df8-408e-a76d-1c51ba1d1e1e None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:28.473366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6af6d-0937-452c-971b-1e0aa77398e3 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:28.537676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adb6b9-52bc-429b-b9f0-4685e28e6fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:28.538515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adb6b9-52bc-429b-b9f0-4685e28e6fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:28.538629 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adb6b9-52bc-429b-b9f0-4685e28e6fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:28.538857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adb6b9-52bc-429b-b9f0-4685e28e6fa8 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:28.539480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00adb6b9-52bc-429b-b9f0-4685e28e6fa8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:28.539786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00adb6b9-52bc-429b-b9f0-4685e28e6fa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:28.541201 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00adb6b9-52bc-429b-b9f0-4685e28e6fa8 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:28.546212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 374/761] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 08:25:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:28.548724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49448b6b-7df8-408e-a76d-1c51ba1d1e1e None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:28.552156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851d849-e879-4e98-a134-22b9f2907421 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:28.552305 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851d849-e879-4e98-a134-22b9f2907421 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:28.552507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851d849-e879-4e98-a134-22b9f2907421 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:28.553996 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 375/762] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:28.554584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851d849-e879-4e98-a134-22b9f2907421 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:28.602277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363ae86-9db0-4ff8-90a2-76699b56c7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:28.603043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363ae86-9db0-4ff8-90a2-76699b56c7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:28.603449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363ae86-9db0-4ff8-90a2-76699b56c7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:28.603781 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363ae86-9db0-4ff8-90a2-76699b56c7f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:28.604459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363ae86-9db0-4ff8-90a2-76699b56c7f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:28.604766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363ae86-9db0-4ff8-90a2-76699b56c7f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:28.606225 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363ae86-9db0-4ff8-90a2-76699b56c7f4 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:28.613379 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:28.627367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 388/763] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:28.627367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363ae86-9db0-4ff8-90a2-76699b56c7f4 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:28.697206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:28.705408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851d849-e879-4e98-a134-22b9f2907421 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:28.748712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfaae8-677e-4737-be81-301d41d9b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:28.749373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfaae8-677e-4737-be81-301d41d9b626 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:28.749652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfaae8-677e-4737-be81-301d41d9b626 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:28.749954 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfaae8-677e-4737-be81-301d41d9b626 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:28.751245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfaae8-677e-4737-be81-301d41d9b626 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:28.751556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfaae8-677e-4737-be81-301d41d9b626 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:28.752988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfaae8-677e-4737-be81-301d41d9b626 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:28.760353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363ae86-9db0-4ff8-90a2-76699b56c7f4 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:28.772662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 376/764] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 08:25:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:28.773977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07806b-d491-4a64-b75c-ec4d85776a38 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:28.774275 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07806b-d491-4a64-b75c-ec4d85776a38 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:28.774533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07806b-d491-4a64-b75c-ec4d85776a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:28.795877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-6f07806b-d491-4a64-b75c-ec4d85776a38 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:28.798193 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6f07806b-d491-4a64-b75c-ec4d85776a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:28.805464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef869bd1-9b1e-49e6-b9aa-af9906e9577d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:28.806162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef869bd1-9b1e-49e6-b9aa-af9906e9577d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:28.806477 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef869bd1-9b1e-49e6-b9aa-af9906e9577d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:28.806802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef869bd1-9b1e-49e6-b9aa-af9906e9577d None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:28.807539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef869bd1-9b1e-49e6-b9aa-af9906e9577d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:28.807877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef869bd1-9b1e-49e6-b9aa-af9906e9577d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:28.809579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef869bd1-9b1e-49e6-b9aa-af9906e9577d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:28.816808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 389/765] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 08:25:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:28.822214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef869bd1-9b1e-49e6-b9aa-af9906e9577d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:28.903989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 377/766] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:28.905312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07806b-d491-4a64-b75c-ec4d85776a38 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:28.948590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:28.957906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef869bd1-9b1e-49e6-b9aa-af9906e9577d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:28.974916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:29.009718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0e706-4cd4-4ea0-9af0-1aeadd4c0371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:29.010740 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0e706-4cd4-4ea0-9af0-1aeadd4c0371 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:29.011202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0e706-4cd4-4ea0-9af0-1aeadd4c0371 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:29.011658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0e706-4cd4-4ea0-9af0-1aeadd4c0371 None admin] PATH_INFO: `/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/9c3c6efc3c154ff89066035cd9583459/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:29.014529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0e706-4cd4-4ea0-9af0-1aeadd4c0371 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca4d4801f2f40419f6881984fbbfbcd, user_id=9c3c6efc3c154ff89066035cd9583459, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:29.016855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0e706-4cd4-4ea0-9af0-1aeadd4c0371 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:29.036999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 390/767] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:25:28 2022] PUT /identity/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/9c3c6efc3c154ff89066035cd9583459/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:25:29.038677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0e706-4cd4-4ea0-9af0-1aeadd4c0371 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:29.050598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07806b-d491-4a64-b75c-ec4d85776a38 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:29.121646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82b940-5b68-4dfd-8088-1ffed517e25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:29.124306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82b940-5b68-4dfd-8088-1ffed517e25c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:29.126005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82b940-5b68-4dfd-8088-1ffed517e25c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:29.128514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82b940-5b68-4dfd-8088-1ffed517e25c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:29.256730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0e706-4cd4-4ea0-9af0-1aeadd4c0371 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:29.269082 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82b940-5b68-4dfd-8088-1ffed517e25c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:29.271025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82b940-5b68-4dfd-8088-1ffed517e25c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:29.309516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a69239-45bd-4990-a466-07a0815c15cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:29.310203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a69239-45bd-4990-a466-07a0815c15cd None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:29.310480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a69239-45bd-4990-a466-07a0815c15cd None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:29.310746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a69239-45bd-4990-a466-07a0815c15cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:29.311455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a69239-45bd-4990-a466-07a0815c15cd None admin] RBAC: Adding query filter params (interface=public, service_id=a977f56e81d34a15a5563d6f085fd42a, region_id=RegionOne) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:29.311757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a69239-45bd-4990-a466-07a0815c15cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:29.313408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:29.314397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a69239-45bd-4990-a466-07a0815c15cd None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:29.330543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 391/768] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 08:25:29 2022] GET /identity/v3/endpoints?service_id=a977f56e81d34a15a5563d6f085fd42a&interface=public®ion_id=RegionOne => generated 194 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:29.330543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a69239-45bd-4990-a466-07a0815c15cd None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:29.375988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:29.438734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 378/769] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:25:29.440406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82b940-5b68-4dfd-8088-1ffed517e25c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:29.473602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a69239-45bd-4990-a466-07a0815c15cd None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:29.529219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935056a2-e8c4-487e-a755-4d4c204c656c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:29.530218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935056a2-e8c4-487e-a755-4d4c204c656c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:29.530509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935056a2-e8c4-487e-a755-4d4c204c656c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:29.530813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935056a2-e8c4-487e-a755-4d4c204c656c None admin] PATH_INFO: `/v3/services/network` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:29.531908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935056a2-e8c4-487e-a755-4d4c204c656c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:29.533329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935056a2-e8c4-487e-a755-4d4c204c656c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-935056a2-e8c4-487e-a755-4d4c204c656c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:29.544448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:29.546360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 08:25:29.548100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:29.551505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 392/770] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:29 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:29.552576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935056a2-e8c4-487e-a755-4d4c204c656c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:29.617117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82b940-5b68-4dfd-8088-1ffed517e25c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:29.652501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:29.666354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08800614-6f39-4fb4-ac95-ee12951a968c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:29.667603 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08800614-6f39-4fb4-ac95-ee12951a968c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:29.668081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08800614-6f39-4fb4-ac95-ee12951a968c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:29.669049 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08800614-6f39-4fb4-ac95-ee12951a968c None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:29.670114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08800614-6f39-4fb4-ac95-ee12951a968c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:29.671855 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08800614-6f39-4fb4-ac95-ee12951a968c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:29.684214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 379/771] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 08:25:29 2022] GET /identity/v3/services => generated 772 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:29.684214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08800614-6f39-4fb4-ac95-ee12951a968c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:29.750540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:29.764173 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935056a2-e8c4-487e-a755-4d4c204c656c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:29.876988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08800614-6f39-4fb4-ac95-ee12951a968c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:29.905938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e1350-dd6d-4b75-949d-46eed7c055ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:29.907281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e1350-dd6d-4b75-949d-46eed7c055ef None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:29.908163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e1350-dd6d-4b75-949d-46eed7c055ef None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:29.908163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e1350-dd6d-4b75-949d-46eed7c055ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:29.928023 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e1350-dd6d-4b75-949d-46eed7c055ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:29.930042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e1350-dd6d-4b75-949d-46eed7c055ef None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:29.935171 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82581345-9edf-4581-90d2-084cc754328c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:29.936390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82581345-9edf-4581-90d2-084cc754328c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:29.936939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82581345-9edf-4581-90d2-084cc754328c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:29.937585 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82581345-9edf-4581-90d2-084cc754328c None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:29.938610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82581345-9edf-4581-90d2-084cc754328c None admin] RBAC: Adding query filter params (name=network) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:29.939553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82581345-9edf-4581-90d2-084cc754328c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:29.941142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82581345-9edf-4581-90d2-084cc754328c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-207e1350-dd6d-4b75-949d-46eed7c055ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:29.948658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:29.949712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:29.950580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:29.950580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:29.950580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:29.950755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 380/772] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:25:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:29.950828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 393/773] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:29.951199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de690670-42c9-4c22-88d0-9ba7af22514c None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:29.951320 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de690670-42c9-4c22-88d0-9ba7af22514c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:29.951593 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de690670-42c9-4c22-88d0-9ba7af22514c None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:29.952829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 381/774] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:29.961039 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d03ec-182c-4993-a312-30f211c2f3c0 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:29.961039 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d03ec-182c-4993-a312-30f211c2f3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:29.961039 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d03ec-182c-4993-a312-30f211c2f3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:29.962890 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de690670-42c9-4c22-88d0-9ba7af22514c None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:29.980473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e15d03ec-182c-4993-a312-30f211c2f3c0 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:29.982143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e15d03ec-182c-4993-a312-30f211c2f3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:30.043287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:30.061235 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 394/775] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:30.062133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d03ec-182c-4993-a312-30f211c2f3c0 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:30.141220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de690670-42c9-4c22-88d0-9ba7af22514c None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:30.215360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:30.234575 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872af70-84cc-431b-8f22-f359aec2a85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:30.235560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872af70-84cc-431b-8f22-f359aec2a85f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:30.236039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872af70-84cc-431b-8f22-f359aec2a85f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:30.236547 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872af70-84cc-431b-8f22-f359aec2a85f None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:30.237457 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872af70-84cc-431b-8f22-f359aec2a85f None admin] RBAC: Adding query filter params (type=network) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:30.237913 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872af70-84cc-431b-8f22-f359aec2a85f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:30.239459 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872af70-84cc-431b-8f22-f359aec2a85f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:30.252323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 382/776] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:25:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:30.258517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872af70-84cc-431b-8f22-f359aec2a85f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:30.259820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d03ec-182c-4993-a312-30f211c2f3c0 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:30.325202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c97981-54a0-419d-95e4-2c64ec0c7492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:30.325991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c97981-54a0-419d-95e4-2c64ec0c7492 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:30.326388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c97981-54a0-419d-95e4-2c64ec0c7492 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:30.326766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c97981-54a0-419d-95e4-2c64ec0c7492 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:30.327522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c97981-54a0-419d-95e4-2c64ec0c7492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:30.327928 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c97981-54a0-419d-95e4-2c64ec0c7492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:30.329507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c97981-54a0-419d-95e4-2c64ec0c7492 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:30.342066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 395/777] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:30.350197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c97981-54a0-419d-95e4-2c64ec0c7492 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:30.414373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872af70-84cc-431b-8f22-f359aec2a85f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:30.435606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:30.467681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10814c9-236f-4466-a355-cb949dfe2ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:30.468501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10814c9-236f-4466-a355-cb949dfe2ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:30.469434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10814c9-236f-4466-a355-cb949dfe2ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:30.469806 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10814c9-236f-4466-a355-cb949dfe2ae5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:30.470829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10814c9-236f-4466-a355-cb949dfe2ae5 None admin] RBAC: Adding query filter params (interface=public, service_id=4341354ed5c444348603cda803b0ed50, region_id=RegionOne) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:30.471251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10814c9-236f-4466-a355-cb949dfe2ae5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:30.472798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10814c9-236f-4466-a355-cb949dfe2ae5 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:30.487505 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 383/778] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 08:25:30 2022] GET /identity/v3/endpoints?service_id=4341354ed5c444348603cda803b0ed50&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:30.488744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10814c9-236f-4466-a355-cb949dfe2ae5 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:30.630861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10814c9-236f-4466-a355-cb949dfe2ae5 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:30.666548 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c97981-54a0-419d-95e4-2c64ec0c7492 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:30.676783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c95592-5dc3-410f-a527-fb2c515d23e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:30.678295 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c95592-5dc3-410f-a527-fb2c515d23e9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:30.678825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c95592-5dc3-410f-a527-fb2c515d23e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:30.679766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c95592-5dc3-410f-a527-fb2c515d23e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:30.695953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c95592-5dc3-410f-a527-fb2c515d23e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-36c95592-5dc3-410f-a527-fb2c515d23e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:30.704782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:30.705835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:30.706662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:30.706662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:30.706662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:30.706662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:30.706662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:30.706662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:30.706662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:30.706662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:30.706662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:30.706662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:30.706662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:25:30.706662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:30.707391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88751bec-bc32-48d0-9b20-9576567cb4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:30.707391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88751bec-bc32-48d0-9b20-9576567cb4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:30.707391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88751bec-bc32-48d0-9b20-9576567cb4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:30.707391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88751bec-bc32-48d0-9b20-9576567cb4d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:30.714990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 384/779] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:25:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:30.714990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c95592-5dc3-410f-a527-fb2c515d23e9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:30.719681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88751bec-bc32-48d0-9b20-9576567cb4d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:30.721153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88751bec-bc32-48d0-9b20-9576567cb4d2 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:30.723241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-88751bec-bc32-48d0-9b20-9576567cb4d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:30.734395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:30.735402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:30.736303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:30.736303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:30.736303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:30.736303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:30.736303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:30.736303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:30.736303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:30.736303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:30.736303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 396/780] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:30.737324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88751bec-bc32-48d0-9b20-9576567cb4d2 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:30.780129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:30.845034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c95592-5dc3-410f-a527-fb2c515d23e9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:30.871279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88751bec-bc32-48d0-9b20-9576567cb4d2 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:30.897677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ae743-7499-4e3c-b368-4ad8e0dfdfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:30.899668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ae743-7499-4e3c-b368-4ad8e0dfdfb6 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:30.900316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ae743-7499-4e3c-b368-4ad8e0dfdfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:30.900934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ae743-7499-4e3c-b368-4ad8e0dfdfb6 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:30.902339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ae743-7499-4e3c-b368-4ad8e0dfdfb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:30.904674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ae743-7499-4e3c-b368-4ad8e0dfdfb6 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:30.911065 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766166ad-83e5-4ca4-b4bd-4ff4b08d72f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:30.911672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766166ad-83e5-4ca4-b4bd-4ff4b08d72f0 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:30.911947 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766166ad-83e5-4ca4-b4bd-4ff4b08d72f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:30.912216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766166ad-83e5-4ca4-b4bd-4ff4b08d72f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:30.912879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766166ad-83e5-4ca4-b4bd-4ff4b08d72f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:30.913337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766166ad-83e5-4ca4-b4bd-4ff4b08d72f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:30.919180 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766166ad-83e5-4ca4-b4bd-4ff4b08d72f0 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:30.919343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 385/781] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 08:25:30 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:30.919343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ae743-7499-4e3c-b368-4ad8e0dfdfb6 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:30.937216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 397/782] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:25:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:30.944048 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766166ad-83e5-4ca4-b4bd-4ff4b08d72f0 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:31.059003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:31.143994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766166ad-83e5-4ca4-b4bd-4ff4b08d72f0 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:31.151368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ae743-7499-4e3c-b368-4ad8e0dfdfb6 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:31.160119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:31.190722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669d222-7c5f-4ecb-9305-5767ab039be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:31.191840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669d222-7c5f-4ecb-9305-5767ab039be4 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:31.192081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dfd0b-8eb2-41a2-a90d-920cef20d32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:31.192434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669d222-7c5f-4ecb-9305-5767ab039be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:31.192851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669d222-7c5f-4ecb-9305-5767ab039be4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:31.192976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dfd0b-8eb2-41a2-a90d-920cef20d32a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:31.193412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dfd0b-8eb2-41a2-a90d-920cef20d32a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:31.194004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dfd0b-8eb2-41a2-a90d-920cef20d32a None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:31.194860 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dfd0b-8eb2-41a2-a90d-920cef20d32a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:31.195351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dfd0b-8eb2-41a2-a90d-920cef20d32a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:31.197138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dfd0b-8eb2-41a2-a90d-920cef20d32a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:31.206867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669d222-7c5f-4ecb-9305-5767ab039be4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:31.208575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669d222-7c5f-4ecb-9305-5767ab039be4 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:31.212992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 386/783] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:31.219972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dfd0b-8eb2-41a2-a90d-920cef20d32a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-9669d222-7c5f-4ecb-9305-5767ab039be4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:31.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:31.228649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:31.229624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:31.229624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:31.229624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:31.232604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 398/784] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:31.237240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669d222-7c5f-4ecb-9305-5767ab039be4 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:31.399832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669d222-7c5f-4ecb-9305-5767ab039be4 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:31.420296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dfd0b-8eb2-41a2-a90d-920cef20d32a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:31.457489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:31.469837 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9491e4-51b6-4639-a4dd-2ab29ad031ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:31.470693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9491e4-51b6-4639-a4dd-2ab29ad031ed None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:31.471029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9491e4-51b6-4639-a4dd-2ab29ad031ed None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:31.471442 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9491e4-51b6-4639-a4dd-2ab29ad031ed None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:31.472242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9491e4-51b6-4639-a4dd-2ab29ad031ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:31.473303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9491e4-51b6-4639-a4dd-2ab29ad031ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:31.474897 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9491e4-51b6-4639-a4dd-2ab29ad031ed None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:31.489643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 399/785] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:31.495792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9491e4-51b6-4639-a4dd-2ab29ad031ed None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:31.525129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8734d-4d8a-4f8b-8846-7d21d7b56dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:31.525469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8734d-4d8a-4f8b-8846-7d21d7b56dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:31.525671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8734d-4d8a-4f8b-8846-7d21d7b56dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:31.529115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8734d-4d8a-4f8b-8846-7d21d7b56dd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:31.529115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8734d-4d8a-4f8b-8846-7d21d7b56dd6 None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=2dae32845ef94e95a9d8f7c16c73c787) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:31.529115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8734d-4d8a-4f8b-8846-7d21d7b56dd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:31.529115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8734d-4d8a-4f8b-8846-7d21d7b56dd6 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:31.538845 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 387/786] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:31 2022] GET /identity/v3/role_assignments?user.id=2dae32845ef94e95a9d8f7c16c73c787&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:31.541306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f78c1e-8fea-46b0-93f6-0c625cffe70a None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:31.541708 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f78c1e-8fea-46b0-93f6-0c625cffe70a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:31.542115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f78c1e-8fea-46b0-93f6-0c625cffe70a None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:31.544424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 388/787] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:31.550892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f03d91-5cd9-4746-9607-dd5d99ae0610 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:31.551409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f03d91-5cd9-4746-9607-dd5d99ae0610 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:31.551780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f03d91-5cd9-4746-9607-dd5d99ae0610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:31.575391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-e8f03d91-5cd9-4746-9607-dd5d99ae0610 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:31.576898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e8f03d91-5cd9-4746-9607-dd5d99ae0610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:31.614043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:31.716586 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9491e4-51b6-4639-a4dd-2ab29ad031ed None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:31.747036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 389/788] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:31.756199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02fb45-2e6e-4833-8672-5028bc0b63f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:31.756866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02fb45-2e6e-4833-8672-5028bc0b63f3 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:31.757187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02fb45-2e6e-4833-8672-5028bc0b63f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:31.757471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02fb45-2e6e-4833-8672-5028bc0b63f3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:31.767187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02fb45-2e6e-4833-8672-5028bc0b63f3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:31.768654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02fb45-2e6e-4833-8672-5028bc0b63f3 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-8b02fb45-2e6e-4833-8672-5028bc0b63f3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:31.781962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:31.782944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:31.783854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:31.783854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:31.783854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:31.783854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:31.783854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:31.783854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:31.783854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:31.783854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:31.783854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 08:25:31.783854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:31.783854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 400/789] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:25:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:31.789152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f03d91-5cd9-4746-9607-dd5d99ae0610 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:31.912499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f03d91-5cd9-4746-9607-dd5d99ae0610 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:31.957214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9bbecc-4d71-4fdf-8a7a-6272264fa348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:31.958268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9bbecc-4d71-4fdf-8a7a-6272264fa348 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:31.958766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9bbecc-4d71-4fdf-8a7a-6272264fa348 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:31.959262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9bbecc-4d71-4fdf-8a7a-6272264fa348 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:31.960208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9bbecc-4d71-4fdf-8a7a-6272264fa348 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:31.960712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9bbecc-4d71-4fdf-8a7a-6272264fa348 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:31.962376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9bbecc-4d71-4fdf-8a7a-6272264fa348 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:31.986936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 390/790] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 08:25:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:31.992602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02fb45-2e6e-4833-8672-5028bc0b63f3 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:32.047540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:32.107203 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9bbecc-4d71-4fdf-8a7a-6272264fa348 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:32.135130 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:32.162825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02fb45-2e6e-4833-8672-5028bc0b63f3 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:32.220537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4ede4-7797-4e19-b180-0be12c58e024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:32.221498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4ede4-7797-4e19-b180-0be12c58e024 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:32.221779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4ede4-7797-4e19-b180-0be12c58e024 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:32.222052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4ede4-7797-4e19-b180-0be12c58e024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:32.235921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4ede4-7797-4e19-b180-0be12c58e024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:32.237478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4ede4-7797-4e19-b180-0be12c58e024 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-1da4ede4-7797-4e19-b180-0be12c58e024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:32.266939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:32.268471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:32.269920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:32.269920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:32.269920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:32.269920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 401/791] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:32.270690 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d9a66-1266-4734-aa53-0f5c59bbf807 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:32.270690 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d9a66-1266-4734-aa53-0f5c59bbf807 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:32.271018 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d9a66-1266-4734-aa53-0f5c59bbf807 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:32.276811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 402/792] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:32.278205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9bbecc-4d71-4fdf-8a7a-6272264fa348 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:32.278419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7a082-878f-4885-bc1a-a233bdb639af None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:32.278604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7a082-878f-4885-bc1a-a233bdb639af None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:32.278835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7a082-878f-4885-bc1a-a233bdb639af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:32.305710 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-a4b7a082-878f-4885-bc1a-a233bdb639af None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:32.307068 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a4b7a082-878f-4885-bc1a-a233bdb639af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:32.335474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cef79f-8da2-4807-b317-c91f17e68f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:32.336228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cef79f-8da2-4807-b317-c91f17e68f83 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:32.336510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cef79f-8da2-4807-b317-c91f17e68f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:32.336781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cef79f-8da2-4807-b317-c91f17e68f83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:32.351079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cef79f-8da2-4807-b317-c91f17e68f83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-11cef79f-8da2-4807-b317-c91f17e68f83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:32.357381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:32.360113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 391/793] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:25:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:32.362728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cef79f-8da2-4807-b317-c91f17e68f83 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:32.398219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 403/794] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:32.399133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7a082-878f-4885-bc1a-a233bdb639af None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:32.417323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:32.516182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:32.531705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7a082-878f-4885-bc1a-a233bdb639af None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:32.573320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca64fea-bb2f-4711-8284-8fe902544348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:32.573954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca64fea-bb2f-4711-8284-8fe902544348 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:32.574238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca64fea-bb2f-4711-8284-8fe902544348 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:32.574510 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca64fea-bb2f-4711-8284-8fe902544348 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:32.575385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca64fea-bb2f-4711-8284-8fe902544348 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:32.575718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca64fea-bb2f-4711-8284-8fe902544348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:32.581120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca64fea-bb2f-4711-8284-8fe902544348 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:32.585565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cef79f-8da2-4807-b317-c91f17e68f83 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:32.593123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 404/795] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:25:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:32.594289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca64fea-bb2f-4711-8284-8fe902544348 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:32.690077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aee04a-3651-411a-b12b-eadc4984dc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:32.691145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aee04a-3651-411a-b12b-eadc4984dc5a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:32.691760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aee04a-3651-411a-b12b-eadc4984dc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:32.692260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aee04a-3651-411a-b12b-eadc4984dc5a None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:32.693626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aee04a-3651-411a-b12b-eadc4984dc5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:32.694347 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aee04a-3651-411a-b12b-eadc4984dc5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:32.705126 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aee04a-3651-411a-b12b-eadc4984dc5a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:32.708231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:32.726460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca64fea-bb2f-4711-8284-8fe902544348 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:32.735224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 392/796] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:32.742979 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aee04a-3651-411a-b12b-eadc4984dc5a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:32.791851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64d096-f92c-46b4-9a44-a5570f400d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:32.792722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64d096-f92c-46b4-9a44-a5570f400d0b None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:32.793265 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64d096-f92c-46b4-9a44-a5570f400d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:32.793757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64d096-f92c-46b4-9a44-a5570f400d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:32.815579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64d096-f92c-46b4-9a44-a5570f400d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:32.819139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64d096-f92c-46b4-9a44-a5570f400d0b None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-ae64d096-f92c-46b4-9a44-a5570f400d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:32.845916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:32.847421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:32.848635 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:32.848635 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:32.848635 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:32.848635 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 405/797] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:32.850208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64d096-f92c-46b4-9a44-a5570f400d0b None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:32.896574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aee04a-3651-411a-b12b-eadc4984dc5a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:32.955838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0718750-be88-469f-8a94-88557ba6d256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:32.956613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0718750-be88-469f-8a94-88557ba6d256 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:32.956900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0718750-be88-469f-8a94-88557ba6d256 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:32.957476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0718750-be88-469f-8a94-88557ba6d256 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:32.966401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0718750-be88-469f-8a94-88557ba6d256 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:32.968372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0718750-be88-469f-8a94-88557ba6d256 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:32.971208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-f0718750-be88-469f-8a94-88557ba6d256 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:32.988821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:32.989804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:32.990654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:32.990654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:32.990654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:32.990654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:32.990654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:32.990654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:32.990654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:32.990654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:32.991539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 393/798] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:32.992508 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0718750-be88-469f-8a94-88557ba6d256 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:33.001560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64d096-f92c-46b4-9a44-a5570f400d0b None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:33.033417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:33.044048 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49a6e6-6732-489d-b1ed-7b19efee5a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:33.045212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49a6e6-6732-489d-b1ed-7b19efee5a3e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:33.045694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49a6e6-6732-489d-b1ed-7b19efee5a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:33.046187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49a6e6-6732-489d-b1ed-7b19efee5a3e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:33.047062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49a6e6-6732-489d-b1ed-7b19efee5a3e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:33.048645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49a6e6-6732-489d-b1ed-7b19efee5a3e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-6c49a6e6-6732-489d-b1ed-7b19efee5a3e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:33.058961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:33.060059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 08:25:33.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:33.069191 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 406/799] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Sep 28 08:25:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:33.070583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49a6e6-6732-489d-b1ed-7b19efee5a3e None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:33.175668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0718750-be88-469f-8a94-88557ba6d256 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:33.204654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49a6e6-6732-489d-b1ed-7b19efee5a3e None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:33.231037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa669894-b0ba-4a14-a266-99010168a4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:33.232125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa669894-b0ba-4a14-a266-99010168a4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:33.232633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa669894-b0ba-4a14-a266-99010168a4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:33.233186 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa669894-b0ba-4a14-a266-99010168a4c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:33.234440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa669894-b0ba-4a14-a266-99010168a4c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:33.235102 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa669894-b0ba-4a14-a266-99010168a4c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:33.237764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa669894-b0ba-4a14-a266-99010168a4c0 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:33.247263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae713b5-a932-4875-95db-0c6a9eb5184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:33.248238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae713b5-a932-4875-95db-0c6a9eb5184b None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:33.248739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae713b5-a932-4875-95db-0c6a9eb5184b None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:33.249242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae713b5-a932-4875-95db-0c6a9eb5184b None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:33.250202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae713b5-a932-4875-95db-0c6a9eb5184b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:33.250734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae713b5-a932-4875-95db-0c6a9eb5184b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:33.252514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae713b5-a932-4875-95db-0c6a9eb5184b None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:33.261196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 394/800] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:33.262852 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa669894-b0ba-4a14-a266-99010168a4c0 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:33.268152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 407/801] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:33.269461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae713b5-a932-4875-95db-0c6a9eb5184b None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:33.333958 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:33.381645 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:33.399670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae713b5-a932-4875-95db-0c6a9eb5184b None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:33.450334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea42b9-4c61-4f2b-906f-bec744495d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:33.451240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea42b9-4c61-4f2b-906f-bec744495d60 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:33.451664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea42b9-4c61-4f2b-906f-bec744495d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:33.452076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea42b9-4c61-4f2b-906f-bec744495d60 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:33.453802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa669894-b0ba-4a14-a266-99010168a4c0 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:33.473450 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea42b9-4c61-4f2b-906f-bec744495d60 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:33.476745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea42b9-4c61-4f2b-906f-bec744495d60 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-95ea42b9-4c61-4f2b-906f-bec744495d60 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:33.491726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:33.492730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:33.493726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:33.493726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:33.493726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:33.493726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:33.493726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:33.493726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:33.493726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:33.493726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:33.493726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:25:33.493726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:33.496888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 408/802] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 08:25:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:33.498004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea42b9-4c61-4f2b-906f-bec744495d60 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:33.537264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912b065-70ef-44b8-be49-c58b2b1120cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:33.539165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b065-70ef-44b8-be49-c58b2b1120cb None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:33.539940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b065-70ef-44b8-be49-c58b2b1120cb None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:33.540724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b065-70ef-44b8-be49-c58b2b1120cb None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:33.542387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912b065-70ef-44b8-be49-c58b2b1120cb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:33.543706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912b065-70ef-44b8-be49-c58b2b1120cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:33.547694 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912b065-70ef-44b8-be49-c58b2b1120cb None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:33.558440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 395/803] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 08:25:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:33.560398 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912b065-70ef-44b8-be49-c58b2b1120cb None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:33.681964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea42b9-4c61-4f2b-906f-bec744495d60 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:33.717788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:33.759288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912b065-70ef-44b8-be49-c58b2b1120cb None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:33.761255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4b619-e6e1-41f2-90c4-6b3f0c87503d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:33.763697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4b619-e6e1-41f2-90c4-6b3f0c87503d None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:33.764652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4b619-e6e1-41f2-90c4-6b3f0c87503d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:33.765529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4b619-e6e1-41f2-90c4-6b3f0c87503d None admin] PATH_INFO: `/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/2f410e0148c1403098b9b73735425726/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:33.769754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4b619-e6e1-41f2-90c4-6b3f0c87503d None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca4d4801f2f40419f6881984fbbfbcd, user_id=2f410e0148c1403098b9b73735425726, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:33.773255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4b619-e6e1-41f2-90c4-6b3f0c87503d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:33.802996 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 409/804] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:25:33 2022] PUT /identity/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/2f410e0148c1403098b9b73735425726/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:25:33.804010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4b619-e6e1-41f2-90c4-6b3f0c87503d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:33.810167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:33.858769 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82d7cb-93fd-4863-90f5-908ef8640e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:33.859962 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82d7cb-93fd-4863-90f5-908ef8640e6a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:33.860507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82d7cb-93fd-4863-90f5-908ef8640e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:33.860787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82d7cb-93fd-4863-90f5-908ef8640e6a None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:33.861859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82d7cb-93fd-4863-90f5-908ef8640e6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:33.862180 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82d7cb-93fd-4863-90f5-908ef8640e6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:33.864548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82d7cb-93fd-4863-90f5-908ef8640e6a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:33.909874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 396/805] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 08:25:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:33.915393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82d7cb-93fd-4863-90f5-908ef8640e6a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:34.054945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4b619-e6e1-41f2-90c4-6b3f0c87503d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:34.106415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ff68f-b766-4b3c-850c-230f3576612d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:34.107923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ff68f-b766-4b3c-850c-230f3576612d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:34.108258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ff68f-b766-4b3c-850c-230f3576612d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:34.108614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ff68f-b766-4b3c-850c-230f3576612d None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:34.110015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ff68f-b766-4b3c-850c-230f3576612d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:34.110415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ff68f-b766-4b3c-850c-230f3576612d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:34.111953 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ff68f-b766-4b3c-850c-230f3576612d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:34.112839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82d7cb-93fd-4863-90f5-908ef8640e6a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:34.120164 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 410/806] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 08:25:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:34.121689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa748e-68e2-4eb0-8cd9-32123ffcaf77 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:34.127464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa748e-68e2-4eb0-8cd9-32123ffcaf77 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:34.127970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa748e-68e2-4eb0-8cd9-32123ffcaf77 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:34.130054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 411/807] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:34.131805 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa748e-68e2-4eb0-8cd9-32123ffcaf77 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:34.176408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c8be0-d98f-4267-8894-37d302d84e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:34.177181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c8be0-d98f-4267-8894-37d302d84e5a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:34.177530 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c8be0-d98f-4267-8894-37d302d84e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:34.177927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c8be0-d98f-4267-8894-37d302d84e5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:34.188083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:34.244538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:34.265384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa748e-68e2-4eb0-8cd9-32123ffcaf77 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:34.271549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c8be0-d98f-4267-8894-37d302d84e5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:34.273030 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c8be0-d98f-4267-8894-37d302d84e5a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:34.308945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed91747-8394-4c2d-9725-7fb26b081869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:34.309952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed91747-8394-4c2d-9725-7fb26b081869 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:34.310399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed91747-8394-4c2d-9725-7fb26b081869 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:34.310806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed91747-8394-4c2d-9725-7fb26b081869 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:34.311589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed91747-8394-4c2d-9725-7fb26b081869 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:34.314233 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed91747-8394-4c2d-9725-7fb26b081869 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:34.348539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 412/808] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:25:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:25:34.351088 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b5398-7c89-42b2-a83f-4753eec3f8da None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:34.351738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b5398-7c89-42b2-a83f-4753eec3f8da None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:34.352324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b5398-7c89-42b2-a83f-4753eec3f8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:34.394061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-313b5398-7c89-42b2-a83f-4753eec3f8da None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:34.397696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-313b5398-7c89-42b2-a83f-4753eec3f8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:34.439265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 397/809] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:25:34.441199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498667b-ea25-4394-8c42-1fbd30226698 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:34.441604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498667b-ea25-4394-8c42-1fbd30226698 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:34.441946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498667b-ea25-4394-8c42-1fbd30226698 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:34.448586 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 398/810] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:34.450852 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498667b-ea25-4394-8c42-1fbd30226698 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:34.570681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 413/811] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:34.573549 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e92dc-d8f5-4611-9bfa-895f9c951f69 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:34.574503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e92dc-d8f5-4611-9bfa-895f9c951f69 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:34.575375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e92dc-d8f5-4611-9bfa-895f9c951f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:34.601083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-462e92dc-d8f5-4611-9bfa-895f9c951f69 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:34.602408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-462e92dc-d8f5-4611-9bfa-895f9c951f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:34.627487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:34.752642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 414/812] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:34.781227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:34.829571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498667b-ea25-4394-8c42-1fbd30226698 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:34.917700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6978f0-6102-4f55-97d1-8eae0e7a4f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:34.918871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6978f0-6102-4f55-97d1-8eae0e7a4f8b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:34.919354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6978f0-6102-4f55-97d1-8eae0e7a4f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:34.919806 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6978f0-6102-4f55-97d1-8eae0e7a4f8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:34.945017 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6978f0-6102-4f55-97d1-8eae0e7a4f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:34.947319 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6978f0-6102-4f55-97d1-8eae0e7a4f8b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-9f6978f0-6102-4f55-97d1-8eae0e7a4f8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:34.975448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:34.976460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:34.977381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:34.977381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:34.977381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:34.978052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 399/813] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 528 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:34.983136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e92dc-d8f5-4611-9bfa-895f9c951f69 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:35.038472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6978f0-6102-4f55-97d1-8eae0e7a4f8b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:35.130216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e92dc-d8f5-4611-9bfa-895f9c951f69 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:35.204724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80607de1-4943-40aa-9f57-b35c6b0a6b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:35.206319 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80607de1-4943-40aa-9f57-b35c6b0a6b70 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:35.207136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80607de1-4943-40aa-9f57-b35c6b0a6b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:35.207855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80607de1-4943-40aa-9f57-b35c6b0a6b70 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:35.209403 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80607de1-4943-40aa-9f57-b35c6b0a6b70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:35.210262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80607de1-4943-40aa-9f57-b35c6b0a6b70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:35.213906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80607de1-4943-40aa-9f57-b35c6b0a6b70 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:35.236406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6978f0-6102-4f55-97d1-8eae0e7a4f8b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:35.238844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 415/814] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:35.240563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80607de1-4943-40aa-9f57-b35c6b0a6b70 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:35.259146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:35.286330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:35.297477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b820cf2-577e-42e9-baa8-5f4b2c0cf3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:35.298630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b820cf2-577e-42e9-baa8-5f4b2c0cf3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:35.299122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b820cf2-577e-42e9-baa8-5f4b2c0cf3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:35.299595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b820cf2-577e-42e9-baa8-5f4b2c0cf3c8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:35.300613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b820cf2-577e-42e9-baa8-5f4b2c0cf3c8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:35.302115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b820cf2-577e-42e9-baa8-5f4b2c0cf3c8 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-4b820cf2-577e-42e9-baa8-5f4b2c0cf3c8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:35.319168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:35.320227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 08:25:35.321050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:35.325630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 400/815] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:35 2022] GET /identity/v3/services/network => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:35.327264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b820cf2-577e-42e9-baa8-5f4b2c0cf3c8 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:35.383242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80607de1-4943-40aa-9f57-b35c6b0a6b70 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:35.428608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58032c5d-a150-4668-8969-8c1643b402b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:35.429984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58032c5d-a150-4668-8969-8c1643b402b8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:35.430476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58032c5d-a150-4668-8969-8c1643b402b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:35.430942 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58032c5d-a150-4668-8969-8c1643b402b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:35.446546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58032c5d-a150-4668-8969-8c1643b402b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-58032c5d-a150-4668-8969-8c1643b402b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:35.454323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:35.455339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:35.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:35.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:35.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:35.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:35.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:35.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:35.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:35.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:35.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:35.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:35.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:35.456160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:35.464331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 416/816] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:35.464331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58032c5d-a150-4668-8969-8c1643b402b8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:35.533970 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b820cf2-577e-42e9-baa8-5f4b2c0cf3c8 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:35.577994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:35.592736 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e0443-6876-428c-87ab-b3cb2a655ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:35.593940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0443-6876-428c-87ab-b3cb2a655ece None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:35.594531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0443-6876-428c-87ab-b3cb2a655ece None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:35.594972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0443-6876-428c-87ab-b3cb2a655ece None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:35.608559 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e0443-6876-428c-87ab-b3cb2a655ece None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:35.609812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58032c5d-a150-4668-8969-8c1643b402b8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:35.612006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e0443-6876-428c-87ab-b3cb2a655ece None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-1d7e0443-6876-428c-87ab-b3cb2a655ece None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:35.635926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:35.636888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:35.637803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:35.637803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:35.637803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:35.637803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:35.637803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:35.637803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:35.637803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:35.637803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:35.638899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 401/817] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:35.639589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e0443-6876-428c-87ab-b3cb2a655ece None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:35.660635 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55fe6af-8235-40e6-8c74-01bbe092e68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:35.662134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fe6af-8235-40e6-8c74-01bbe092e68c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:35.662630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fe6af-8235-40e6-8c74-01bbe092e68c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:35.663362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fe6af-8235-40e6-8c74-01bbe092e68c None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:35.664244 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55fe6af-8235-40e6-8c74-01bbe092e68c None admin] RBAC: Adding query filter params (name=network) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:35.664810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55fe6af-8235-40e6-8c74-01bbe092e68c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:35.667571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55fe6af-8235-40e6-8c74-01bbe092e68c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:35.684528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 417/818] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:25:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:35.685630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55fe6af-8235-40e6-8c74-01bbe092e68c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:35.767156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:35.866343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e0443-6876-428c-87ab-b3cb2a655ece None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:35.880011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55fe6af-8235-40e6-8c74-01bbe092e68c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:35.916441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d582d8f-292a-458b-9a5f-17ea9e4e84a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:35.917061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d582d8f-292a-458b-9a5f-17ea9e4e84a5 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:35.917590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d582d8f-292a-458b-9a5f-17ea9e4e84a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:35.918099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d582d8f-292a-458b-9a5f-17ea9e4e84a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:35.919232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d582d8f-292a-458b-9a5f-17ea9e4e84a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:35.919765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d582d8f-292a-458b-9a5f-17ea9e4e84a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:35.923111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d582d8f-292a-458b-9a5f-17ea9e4e84a5 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:35.926175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:35.933279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4dbde-dbe3-4aec-b6e8-bb7084b60beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:35.934975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4dbde-dbe3-4aec-b6e8-bb7084b60beb None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:35.935473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4dbde-dbe3-4aec-b6e8-bb7084b60beb None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:35.935944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4dbde-dbe3-4aec-b6e8-bb7084b60beb None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:35.937174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4dbde-dbe3-4aec-b6e8-bb7084b60beb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:35.937814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4dbde-dbe3-4aec-b6e8-bb7084b60beb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:35.939554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4dbde-dbe3-4aec-b6e8-bb7084b60beb None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:35.944552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 402/819] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:35.944552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d582d8f-292a-458b-9a5f-17ea9e4e84a5 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:35.962404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 418/820] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:35.963851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4dbde-dbe3-4aec-b6e8-bb7084b60beb None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:36.093860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4dbde-dbe3-4aec-b6e8-bb7084b60beb None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:36.153392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d582d8f-292a-458b-9a5f-17ea9e4e84a5 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:36.162071 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96534058-533e-4917-b960-e0738bb376d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:36.162721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96534058-533e-4917-b960-e0738bb376d0 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:36.162999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96534058-533e-4917-b960-e0738bb376d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:36.163395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96534058-533e-4917-b960-e0738bb376d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:36.176730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96534058-533e-4917-b960-e0738bb376d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:36.178542 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96534058-533e-4917-b960-e0738bb376d0 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:36.182492 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-96534058-533e-4917-b960-e0738bb376d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:36.200722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:36.205622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:36.207317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:36.207317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:36.207317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:36.209604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 419/821] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:36.211125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96534058-533e-4917-b960-e0738bb376d0 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:36.221438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eabef3-d53b-4ed7-b272-999f9113776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:36.223107 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eabef3-d53b-4ed7-b272-999f9113776b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:36.223992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eabef3-d53b-4ed7-b272-999f9113776b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:36.224802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eabef3-d53b-4ed7-b272-999f9113776b None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:36.226322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eabef3-d53b-4ed7-b272-999f9113776b None admin] RBAC: Adding query filter params (type=network) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:36.227267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eabef3-d53b-4ed7-b272-999f9113776b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:36.230363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eabef3-d53b-4ed7-b272-999f9113776b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:36.245454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 403/822] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:25:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:36.247846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eabef3-d53b-4ed7-b272-999f9113776b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:36.271004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:36.355454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96534058-533e-4917-b960-e0738bb376d0 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:36.398046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e276a5-d812-45ed-986f-d8cd9df1ec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:36.398675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e276a5-d812-45ed-986f-d8cd9df1ec29 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:36.398952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e276a5-d812-45ed-986f-d8cd9df1ec29 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:36.399225 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e276a5-d812-45ed-986f-d8cd9df1ec29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:36.399969 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e276a5-d812-45ed-986f-d8cd9df1ec29 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=9c3c6efc3c154ff89066035cd9583459) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:36.400278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e276a5-d812-45ed-986f-d8cd9df1ec29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:36.403909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e276a5-d812-45ed-986f-d8cd9df1ec29 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:36.421130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 420/823] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:36 2022] GET /identity/v3/role_assignments?user.id=9c3c6efc3c154ff89066035cd9583459&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:36.421130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e276a5-d812-45ed-986f-d8cd9df1ec29 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:36.446612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eabef3-d53b-4ed7-b272-999f9113776b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:36.497033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7edc0b-b24b-470d-adc8-751a55eff61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:36.497800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7edc0b-b24b-470d-adc8-751a55eff61e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:36.498076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7edc0b-b24b-470d-adc8-751a55eff61e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:36.498346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7edc0b-b24b-470d-adc8-751a55eff61e None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:36.498998 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7edc0b-b24b-470d-adc8-751a55eff61e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:36.499297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7edc0b-b24b-470d-adc8-751a55eff61e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:36.500960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7edc0b-b24b-470d-adc8-751a55eff61e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:36.528886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 404/824] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:36.533143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82588186-cddf-4f32-8f88-24172d809c27 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:36.533143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82588186-cddf-4f32-8f88-24172d809c27 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:36.533143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82588186-cddf-4f32-8f88-24172d809c27 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:36.537900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 405/825] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:36.538877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82588186-cddf-4f32-8f88-24172d809c27 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:36.617732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e276a5-d812-45ed-986f-d8cd9df1ec29 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:36.623392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:36.635478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:36.674663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884f03f-610a-493d-8bb3-79f2b6a9612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:36.675659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884f03f-610a-493d-8bb3-79f2b6a9612f None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:36.676455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884f03f-610a-493d-8bb3-79f2b6a9612f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:36.676915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884f03f-610a-493d-8bb3-79f2b6a9612f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:36.677859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884f03f-610a-493d-8bb3-79f2b6a9612f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:36.680987 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884f03f-610a-493d-8bb3-79f2b6a9612f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:36.713282 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 421/826] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:25:36 2022] POST /identity/v3/endpoints => generated 337 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:25:36.715703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987d433-29e2-4bf4-b527-a22a91a624ee None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:36.715999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987d433-29e2-4bf4-b527-a22a91a624ee None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:36.716276 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987d433-29e2-4bf4-b527-a22a91a624ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:36.742729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e987d433-29e2-4bf4-b527-a22a91a624ee None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:36.744032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e987d433-29e2-4bf4-b527-a22a91a624ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:36.765125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82588186-cddf-4f32-8f88-24172d809c27 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:36.852904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53547a8-ac0e-4a92-b35c-2c4020edce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:36.854002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53547a8-ac0e-4a92-b35c-2c4020edce7a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:36.854429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53547a8-ac0e-4a92-b35c-2c4020edce7a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:36.854838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53547a8-ac0e-4a92-b35c-2c4020edce7a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:36.861304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 422/827] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:36.870875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53547a8-ac0e-4a92-b35c-2c4020edce7a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:36.873768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53547a8-ac0e-4a92-b35c-2c4020edce7a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-e53547a8-ac0e-4a92-b35c-2c4020edce7a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:36.898089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:36.899197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:36.900290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:36.900290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:36.900290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:36.900290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:36.900290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:36.900290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:36.900290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:36.900290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:36.900290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 08:25:36.900290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:36.900891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 406/828] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 08:25:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:36.911023 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987d433-29e2-4bf4-b527-a22a91a624ee None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:37.049887 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:37.059836 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987d433-29e2-4bf4-b527-a22a91a624ee None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:37.109915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cb8f9-6466-4f77-a7bd-827c965e91d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:37.111379 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cb8f9-6466-4f77-a7bd-827c965e91d4 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:37.111879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cb8f9-6466-4f77-a7bd-827c965e91d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:37.112417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cb8f9-6466-4f77-a7bd-827c965e91d4 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:37.113477 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49cb8f9-6466-4f77-a7bd-827c965e91d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:37.114010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49cb8f9-6466-4f77-a7bd-827c965e91d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:37.115776 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49cb8f9-6466-4f77-a7bd-827c965e91d4 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:37.137851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 423/829] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 08:25:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:37.144545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53547a8-ac0e-4a92-b35c-2c4020edce7a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:37.169793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cb8f9-6466-4f77-a7bd-827c965e91d4 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:37.302613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:37.308737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53547a8-ac0e-4a92-b35c-2c4020edce7a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:37.323962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cb8f9-6466-4f77-a7bd-827c965e91d4 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:37.363403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590db767-1837-4956-8522-3fbc702aa927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:37.364079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590db767-1837-4956-8522-3fbc702aa927 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:37.364360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590db767-1837-4956-8522-3fbc702aa927 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:37.364631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590db767-1837-4956-8522-3fbc702aa927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:37.372738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843ac1b-84cb-4c1b-be8a-05685f773c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:37.374485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843ac1b-84cb-4c1b-be8a-05685f773c11 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:37.375518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843ac1b-84cb-4c1b-be8a-05685f773c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:37.376532 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843ac1b-84cb-4c1b-be8a-05685f773c11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:37.378929 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590db767-1837-4956-8522-3fbc702aa927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:37.381118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590db767-1837-4956-8522-3fbc702aa927 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:37.390364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843ac1b-84cb-4c1b-be8a-05685f773c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-d843ac1b-84cb-4c1b-be8a-05685f773c11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:37.396460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:37.397501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:37.398351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:37.398351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:37.398351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:37.398351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:37.398351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:37.398351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:37.398351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:37.398351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:37.398351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:37.398351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:37.398351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:25:37.398351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:37.401076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 424/830] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:25:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-590db767-1837-4956-8522-3fbc702aa927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:37.406176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:37.407246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:37.408132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:37.408132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:37.408132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:37.409780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843ac1b-84cb-4c1b-be8a-05685f773c11 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:37.411558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 407/831] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:37.419918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590db767-1837-4956-8522-3fbc702aa927 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:37.430197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:37.539472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843ac1b-84cb-4c1b-be8a-05685f773c11 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:37.550298 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590db767-1837-4956-8522-3fbc702aa927 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:37.602713 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae283a2-96e6-40ee-8ee7-93488ad9b19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:37.603624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae283a2-96e6-40ee-8ee7-93488ad9b19e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:37.604118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae283a2-96e6-40ee-8ee7-93488ad9b19e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:37.604539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae283a2-96e6-40ee-8ee7-93488ad9b19e None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:37.605553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae283a2-96e6-40ee-8ee7-93488ad9b19e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:37.606050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae283a2-96e6-40ee-8ee7-93488ad9b19e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:37.607818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae283a2-96e6-40ee-8ee7-93488ad9b19e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:37.609763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedc784-2eb1-4abd-bfab-d0978555246d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:37.610394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedc784-2eb1-4abd-bfab-d0978555246d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:37.610667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedc784-2eb1-4abd-bfab-d0978555246d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:37.610930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedc784-2eb1-4abd-bfab-d0978555246d None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:37.611622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedc784-2eb1-4abd-bfab-d0978555246d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:37.611913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedc784-2eb1-4abd-bfab-d0978555246d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:37.613334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedc784-2eb1-4abd-bfab-d0978555246d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:37.632234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 408/832] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:37.632234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849cbea-5f56-48e7-847f-d33129a1411b None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:37.632234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849cbea-5f56-48e7-847f-d33129a1411b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:37.632234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849cbea-5f56-48e7-847f-d33129a1411b None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:37.632234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 409/833] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:37.633508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 425/834] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:25:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:37.636246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849cbea-5f56-48e7-847f-d33129a1411b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:37.641863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc7ebc-96f7-4449-8515-81ded28aac1b None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:37.642202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc7ebc-96f7-4449-8515-81ded28aac1b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:37.642506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc7ebc-96f7-4449-8515-81ded28aac1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:37.653615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:37.667918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5bfc7ebc-96f7-4449-8515-81ded28aac1b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:37.669183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5bfc7ebc-96f7-4449-8515-81ded28aac1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:37.756386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 426/835] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:25:37.757935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc7ebc-96f7-4449-8515-81ded28aac1b None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:37.765922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:37.815425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849cbea-5f56-48e7-847f-d33129a1411b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:37.876859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c205d-ba13-45aa-bfdb-967d1368f52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:37.877861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c205d-ba13-45aa-bfdb-967d1368f52c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:37.878328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c205d-ba13-45aa-bfdb-967d1368f52c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:37.878811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c205d-ba13-45aa-bfdb-967d1368f52c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:37.912238 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c205d-ba13-45aa-bfdb-967d1368f52c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:37.914207 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c205d-ba13-45aa-bfdb-967d1368f52c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-ce2c205d-ba13-45aa-bfdb-967d1368f52c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:37.931324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:37.932344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:37.933258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:37.933258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:37.933258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:37.935190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 410/836] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:37.947955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c205d-ba13-45aa-bfdb-967d1368f52c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:37.958047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc7ebc-96f7-4449-8515-81ded28aac1b None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:38.026336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:38.027151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2711a4-f7b8-45c7-a105-9ccb87bea6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:38.028090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2711a4-f7b8-45c7-a105-9ccb87bea6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:38.028284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2711a4-f7b8-45c7-a105-9ccb87bea6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:38.028546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2711a4-f7b8-45c7-a105-9ccb87bea6d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:38.038448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2711a4-f7b8-45c7-a105-9ccb87bea6d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:38.039911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2711a4-f7b8-45c7-a105-9ccb87bea6d8 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-2e2711a4-f7b8-45c7-a105-9ccb87bea6d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:38.053182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:38.054156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:38.055066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:38.055066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:38.055066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:38.055066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:38.055066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:38.055066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:38.055066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:38.055066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:38.056204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 427/837] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:38.057240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2711a4-f7b8-45c7-a105-9ccb87bea6d8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:38.084837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c205d-ba13-45aa-bfdb-967d1368f52c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:38.126155 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a137b43-7852-46b4-932e-28ea51ad74b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:38.126778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a137b43-7852-46b4-932e-28ea51ad74b2 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:38.127116 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a137b43-7852-46b4-932e-28ea51ad74b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:38.127335 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a137b43-7852-46b4-932e-28ea51ad74b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:38.127988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a137b43-7852-46b4-932e-28ea51ad74b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:38.128292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a137b43-7852-46b4-932e-28ea51ad74b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:38.129786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a137b43-7852-46b4-932e-28ea51ad74b2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:38.144249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 411/838] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:38.145144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a137b43-7852-46b4-932e-28ea51ad74b2 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:38.150425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:38.197176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2711a4-f7b8-45c7-a105-9ccb87bea6d8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:38.241334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741e19-5ff8-4beb-a606-ca4a533e8b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:38.242089 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741e19-5ff8-4beb-a606-ca4a533e8b71 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:38.242428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741e19-5ff8-4beb-a606-ca4a533e8b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:38.242702 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741e19-5ff8-4beb-a606-ca4a533e8b71 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:38.243295 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e741e19-5ff8-4beb-a606-ca4a533e8b71 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:38.244673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e741e19-5ff8-4beb-a606-ca4a533e8b71 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-2e741e19-5ff8-4beb-a606-ca4a533e8b71 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:38.254841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:38.256129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 08:25:38.256973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:38.261001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 428/839] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:25:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:38.261740 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741e19-5ff8-4beb-a606-ca4a533e8b71 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:38.271145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a137b43-7852-46b4-932e-28ea51ad74b2 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:38.298938 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:38.312071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01547d-620e-4f7a-ad48-05025f941572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:38.313628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01547d-620e-4f7a-ad48-05025f941572 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:38.313628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01547d-620e-4f7a-ad48-05025f941572 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:38.313910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01547d-620e-4f7a-ad48-05025f941572 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:38.314519 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01547d-620e-4f7a-ad48-05025f941572 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:38.314835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01547d-620e-4f7a-ad48-05025f941572 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:38.316293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01547d-620e-4f7a-ad48-05025f941572 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:38.333367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 412/840] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:38.334939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01547d-620e-4f7a-ad48-05025f941572 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:38.415121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741e19-5ff8-4beb-a606-ca4a533e8b71 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:38.455867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:38.472723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69394edb-996d-443b-a066-c0cb6dc13372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:38.473784 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69394edb-996d-443b-a066-c0cb6dc13372 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:38.474241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69394edb-996d-443b-a066-c0cb6dc13372 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:38.474699 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69394edb-996d-443b-a066-c0cb6dc13372 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:38.485321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69394edb-996d-443b-a066-c0cb6dc13372 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:38.486773 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69394edb-996d-443b-a066-c0cb6dc13372 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-69394edb-996d-443b-a066-c0cb6dc13372 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:38.499462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:38.502684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:38.504637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:38.504637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:38.504637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:38.504637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:38.504637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:38.504637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:38.504637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:38.504637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:38.504637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 08:25:38.504637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:38.504637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 429/841] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:25:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:38.504637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69394edb-996d-443b-a066-c0cb6dc13372 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:38.552809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01547d-620e-4f7a-ad48-05025f941572 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:38.591959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aed74d-5f58-4e0b-931b-2a8b640a9344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:38.592577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed74d-5f58-4e0b-931b-2a8b640a9344 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:38.592953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed74d-5f58-4e0b-931b-2a8b640a9344 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:38.593340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed74d-5f58-4e0b-931b-2a8b640a9344 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:38.594565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aed74d-5f58-4e0b-931b-2a8b640a9344 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:38.594867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aed74d-5f58-4e0b-931b-2a8b640a9344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:38.596336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aed74d-5f58-4e0b-931b-2a8b640a9344 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:38.602315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 413/842] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:25:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:38.603294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aed74d-5f58-4e0b-931b-2a8b640a9344 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:38.670956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69394edb-996d-443b-a066-c0cb6dc13372 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:38.697687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:38.732553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97209d-9798-4d73-8f0f-90230def5cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:38.733806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97209d-9798-4d73-8f0f-90230def5cce None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:38.735086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97209d-9798-4d73-8f0f-90230def5cce None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:38.735549 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97209d-9798-4d73-8f0f-90230def5cce None admin] PATH_INFO: `/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/2dae32845ef94e95a9d8f7c16c73c787/roles/5354696222554ab2a19a3a839682a597` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:38.767812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97209d-9798-4d73-8f0f-90230def5cce None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca4d4801f2f40419f6881984fbbfbcd, user_id=2dae32845ef94e95a9d8f7c16c73c787, role_id=5354696222554ab2a19a3a839682a597)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:38.770514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97209d-9798-4d73-8f0f-90230def5cce None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:38.774511 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aed74d-5f58-4e0b-931b-2a8b640a9344 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:38.794898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 430/843] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:25:38 2022] PUT /identity/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/2dae32845ef94e95a9d8f7c16c73c787/roles/5354696222554ab2a19a3a839682a597 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:25:38.796421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97209d-9798-4d73-8f0f-90230def5cce None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:38.826758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbeb4db-01b5-4e0b-adf0-d09be2764162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:38.827622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbeb4db-01b5-4e0b-adf0-d09be2764162 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:38.828046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbeb4db-01b5-4e0b-adf0-d09be2764162 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:38.828397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbeb4db-01b5-4e0b-adf0-d09be2764162 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:38.829459 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbeb4db-01b5-4e0b-adf0-d09be2764162 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:38.829883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbeb4db-01b5-4e0b-adf0-d09be2764162 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:38.831616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbeb4db-01b5-4e0b-adf0-d09be2764162 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:38.856308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 414/844] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 08:25:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:38.863413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbeb4db-01b5-4e0b-adf0-d09be2764162 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:38.891798 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:38.977905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97209d-9798-4d73-8f0f-90230def5cce None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:39.023831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586ae84-6047-40c1-8331-b4eac14b6632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:39.024463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586ae84-6047-40c1-8331-b4eac14b6632 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:39.024746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586ae84-6047-40c1-8331-b4eac14b6632 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:39.025019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586ae84-6047-40c1-8331-b4eac14b6632 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:39.025709 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586ae84-6047-40c1-8331-b4eac14b6632 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:39.026011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586ae84-6047-40c1-8331-b4eac14b6632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:39.027940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586ae84-6047-40c1-8331-b4eac14b6632 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:39.033923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 431/845] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:25:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:39.060002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbeb4db-01b5-4e0b-adf0-d09be2764162 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:39.066824 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:39.140031 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e586cd-560d-45f6-b42f-89ec72cab5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:39.141842 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e586cd-560d-45f6-b42f-89ec72cab5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:39.142370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e586cd-560d-45f6-b42f-89ec72cab5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:39.142968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e586cd-560d-45f6-b42f-89ec72cab5e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:39.262867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b1838-b370-4650-bc83-84fd75688402 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:39.263386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b1838-b370-4650-bc83-84fd75688402 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:39.263386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b1838-b370-4650-bc83-84fd75688402 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:39.264716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 432/846] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:39.273350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c67745-c55f-4123-a0be-5a8afbdeb48f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:39.273350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c67745-c55f-4123-a0be-5a8afbdeb48f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:39.273350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c67745-c55f-4123-a0be-5a8afbdeb48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:39.289154 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e586cd-560d-45f6-b42f-89ec72cab5e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:39.290669 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e586cd-560d-45f6-b42f-89ec72cab5e9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:39.296532 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-92c67745-c55f-4123-a0be-5a8afbdeb48f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:39.297941 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-92c67745-c55f-4123-a0be-5a8afbdeb48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:39.393223 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 433/847] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:25:39.419545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 415/848] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:38 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:25:39.428479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c67745-c55f-4123-a0be-5a8afbdeb48f None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:39.482657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:39.619485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c67745-c55f-4123-a0be-5a8afbdeb48f None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:39.674502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420fb30-6772-4822-89e7-362a83e6f465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:39.675220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420fb30-6772-4822-89e7-362a83e6f465 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:39.675502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420fb30-6772-4822-89e7-362a83e6f465 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:39.675802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420fb30-6772-4822-89e7-362a83e6f465 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:39.688395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420fb30-6772-4822-89e7-362a83e6f465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:39.689972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420fb30-6772-4822-89e7-362a83e6f465 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-e420fb30-6772-4822-89e7-362a83e6f465 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:39.707033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:39.708597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:39.709806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:39.709806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:39.709806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:39.709806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 434/849] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:39.718070 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e586cd-560d-45f6-b42f-89ec72cab5e9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:39.725003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:39.763554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420fb30-6772-4822-89e7-362a83e6f465 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:39.895153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:39.916944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420fb30-6772-4822-89e7-362a83e6f465 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:39.919040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e586cd-560d-45f6-b42f-89ec72cab5e9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:39.958422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6879cfc-439c-41f8-b113-85f8950e293e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:39.959078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6879cfc-439c-41f8-b113-85f8950e293e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:39.959358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6879cfc-439c-41f8-b113-85f8950e293e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:39.959637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6879cfc-439c-41f8-b113-85f8950e293e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:39.960242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6879cfc-439c-41f8-b113-85f8950e293e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:39.961767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6879cfc-439c-41f8-b113-85f8950e293e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-d6879cfc-439c-41f8-b113-85f8950e293e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:39.972775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:39.977741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:39.978772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 435/850] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Sep 28 08:25:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:39.985315 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6879cfc-439c-41f8-b113-85f8950e293e None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:39.993373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0eb58e-c703-4fe4-b0b3-077c75511c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:39.994438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eb58e-c703-4fe4-b0b3-077c75511c4e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:39.995189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eb58e-c703-4fe4-b0b3-077c75511c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:39.995939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eb58e-c703-4fe4-b0b3-077c75511c4e None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:39.997453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0eb58e-c703-4fe4-b0b3-077c75511c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:39.998296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0eb58e-c703-4fe4-b0b3-077c75511c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:40.001794 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0eb58e-c703-4fe4-b0b3-077c75511c4e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:40.028897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 416/851] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:40.030865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0eb58e-c703-4fe4-b0b3-077c75511c4e None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:40.124773 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6879cfc-439c-41f8-b113-85f8950e293e None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:40.134892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:40.187888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0eb58e-c703-4fe4-b0b3-077c75511c4e None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:40.191791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf2e0a-73cd-40b8-aa93-30e23713058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:40.192432 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf2e0a-73cd-40b8-aa93-30e23713058e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:40.192710 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf2e0a-73cd-40b8-aa93-30e23713058e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:40.192984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf2e0a-73cd-40b8-aa93-30e23713058e None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:40.193673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf2e0a-73cd-40b8-aa93-30e23713058e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:40.193979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf2e0a-73cd-40b8-aa93-30e23713058e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:40.195387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf2e0a-73cd-40b8-aa93-30e23713058e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:40.205241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 436/852] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 08:25:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:40.209457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf2e0a-73cd-40b8-aa93-30e23713058e None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:40.236729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:40.247050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8841dc-9b5e-4719-b2fd-b14bbb61e579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:40.248231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8841dc-9b5e-4719-b2fd-b14bbb61e579 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:40.248804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8841dc-9b5e-4719-b2fd-b14bbb61e579 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:40.249420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8841dc-9b5e-4719-b2fd-b14bbb61e579 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:40.263222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8841dc-9b5e-4719-b2fd-b14bbb61e579 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:40.266660 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8841dc-9b5e-4719-b2fd-b14bbb61e579 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-ce8841dc-9b5e-4719-b2fd-b14bbb61e579 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:40.289413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:40.290384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:40.291272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:40.291272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:40.291272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:40.291272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:40.291272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:40.291272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:40.291272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:40.291272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:40.297426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 417/853] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:40.302475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8841dc-9b5e-4719-b2fd-b14bbb61e579 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:40.362606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf2e0a-73cd-40b8-aa93-30e23713058e None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:40.420623 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becad321-d97a-48b5-86fa-66aec82839ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:40.421430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becad321-d97a-48b5-86fa-66aec82839ac None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:40.422027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becad321-d97a-48b5-86fa-66aec82839ac None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:40.422582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becad321-d97a-48b5-86fa-66aec82839ac None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:40.423682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becad321-d97a-48b5-86fa-66aec82839ac None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:40.424146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becad321-d97a-48b5-86fa-66aec82839ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:40.425814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becad321-d97a-48b5-86fa-66aec82839ac None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:40.432537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 437/854] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 08:25:40 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:40.459563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8841dc-9b5e-4719-b2fd-b14bbb61e579 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:40.496337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:40.512191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f8e563-9140-4ccb-bb0e-206795196ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:40.513635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8e563-9140-4ccb-bb0e-206795196ccb None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:40.513919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8e563-9140-4ccb-bb0e-206795196ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:40.514245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f8e563-9140-4ccb-bb0e-206795196ccb None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:40.514980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8e563-9140-4ccb-bb0e-206795196ccb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:40.515355 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8e563-9140-4ccb-bb0e-206795196ccb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:40.517804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f8e563-9140-4ccb-bb0e-206795196ccb None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:40.536588 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 418/855] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:40.545565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becad321-d97a-48b5-86fa-66aec82839ac None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:40.656978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:40.685228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becad321-d97a-48b5-86fa-66aec82839ac None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:40.727328 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74985a-1433-4774-bbba-7ca7759c7a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:40.727899 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74985a-1433-4774-bbba-7ca7759c7a3d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:40.728182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74985a-1433-4774-bbba-7ca7759c7a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:40.728457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74985a-1433-4774-bbba-7ca7759c7a3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:40.729261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74985a-1433-4774-bbba-7ca7759c7a3d None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=2f410e0148c1403098b9b73735425726) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:40.729577 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74985a-1433-4774-bbba-7ca7759c7a3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:40.731054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74985a-1433-4774-bbba-7ca7759c7a3d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:40.738891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 438/856] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:40 2022] GET /identity/v3/role_assignments?user.id=2f410e0148c1403098b9b73735425726&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:41.220795 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe17ca-2684-4f37-bcf8-0108a9ee3f1c None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:41.221294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe17ca-2684-4f37-bcf8-0108a9ee3f1c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:41.221685 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe17ca-2684-4f37-bcf8-0108a9ee3f1c None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:41.223420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 419/857] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:41.228719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2381cda-8d43-4719-9513-2df1797e90c6 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:41.229012 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2381cda-8d43-4719-9513-2df1797e90c6 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:41.229298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2381cda-8d43-4719-9513-2df1797e90c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:41.248978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-a2381cda-8d43-4719-9513-2df1797e90c6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:41.250770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a2381cda-8d43-4719-9513-2df1797e90c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:41.329393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 439/858] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:41.575777 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1cb0-146b-4198-8985-214c750bbcc9 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:41.576287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1cb0-146b-4198-8985-214c750bbcc9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:41.576287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b1cb0-146b-4198-8985-214c750bbcc9 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:41.578465 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 420/859] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:41.583815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3d668-cfe3-47d2-9045-8e69545713ec None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:41.584237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3d668-cfe3-47d2-9045-8e69545713ec None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:41.584447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3d668-cfe3-47d2-9045-8e69545713ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:41.603725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-a4a3d668-cfe3-47d2-9045-8e69545713ec None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:41.604936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a4a3d668-cfe3-47d2-9045-8e69545713ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:41.607606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b1cb0-146b-4198-8985-214c750bbcc9 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:41.688831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:41.701567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 440/860] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:41.735012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b1cb0-146b-4198-8985-214c750bbcc9 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:41.776841 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6135b06-a8b5-4178-9647-614b0d87dc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:41.777892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6135b06-a8b5-4178-9647-614b0d87dc48 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:41.778436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6135b06-a8b5-4178-9647-614b0d87dc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:41.778888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6135b06-a8b5-4178-9647-614b0d87dc48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:41.792801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6135b06-a8b5-4178-9647-614b0d87dc48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-d6135b06-a8b5-4178-9647-614b0d87dc48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:41.798609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:41.799737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:41.800751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:41.800751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:41.800751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:41.800751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:41.800751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:41.800751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:41.800751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:41.800751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:41.800751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:41.800751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:41.800751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:25:41.800751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:41.803138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 421/861] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:25:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:41.809061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3d668-cfe3-47d2-9045-8e69545713ec None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:41.914249 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3d668-cfe3-47d2-9045-8e69545713ec None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:41.954873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b1538-04fa-4975-98ee-b609b8d7263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:41.955491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b1538-04fa-4975-98ee-b609b8d7263b None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:41.955770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b1538-04fa-4975-98ee-b609b8d7263b None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:41.956041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b1538-04fa-4975-98ee-b609b8d7263b None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:41.956711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b1538-04fa-4975-98ee-b609b8d7263b None admin] RBAC: Adding query filter params (name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:41.957011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b1538-04fa-4975-98ee-b609b8d7263b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:41.958398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b1538-04fa-4975-98ee-b609b8d7263b None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:41.967219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 441/862] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:25:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:41.973274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6135b06-a8b5-4178-9647-614b0d87dc48 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:42.011238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b1538-04fa-4975-98ee-b609b8d7263b None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:42.041076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:42.084037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:42.107987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6135b06-a8b5-4178-9647-614b0d87dc48 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:42.139969 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b1538-04fa-4975-98ee-b609b8d7263b None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:42.148887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ead55b-f3af-4f53-8989-b040c315444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:42.150026 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ead55b-f3af-4f53-8989-b040c315444f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:42.150026 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ead55b-f3af-4f53-8989-b040c315444f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:42.150432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ead55b-f3af-4f53-8989-b040c315444f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:42.166179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ead55b-f3af-4f53-8989-b040c315444f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:42.168298 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ead55b-f3af-4f53-8989-b040c315444f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:42.184119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b6342-924a-445d-8a06-72837a9f24d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:42.184892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b6342-924a-445d-8a06-72837a9f24d7 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:42.185250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b6342-924a-445d-8a06-72837a9f24d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:42.185591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b6342-924a-445d-8a06-72837a9f24d7 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:42.186320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b6342-924a-445d-8a06-72837a9f24d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:42.187882 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b6342-924a-445d-8a06-72837a9f24d7 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-d6ead55b-f3af-4f53-8989-b040c315444f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:42.191424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:42.192421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:42.193336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:42.193336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:42.193336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:42.195121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 422/863] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:42.201617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ead55b-f3af-4f53-8989-b040c315444f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:42.209637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 442/864] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:25:42 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:25:42.357878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:42.370891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ead55b-f3af-4f53-8989-b040c315444f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:42.455774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58f00d-d69e-4c3f-9248-ebeb039c829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:42.457144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58f00d-d69e-4c3f-9248-ebeb039c829f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:42.457869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58f00d-d69e-4c3f-9248-ebeb039c829f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:42.458583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58f00d-d69e-4c3f-9248-ebeb039c829f None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:42.459988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58f00d-d69e-4c3f-9248-ebeb039c829f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:42.460736 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58f00d-d69e-4c3f-9248-ebeb039c829f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:42.463567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58f00d-d69e-4c3f-9248-ebeb039c829f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:42.480229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 423/865] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:42.486396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b6342-924a-445d-8a06-72837a9f24d7 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:42.664779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b6342-924a-445d-8a06-72837a9f24d7 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:42.711994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:42.737175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4867c-3559-4642-9ff1-1fd9ab58b83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:42.738298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4867c-3559-4642-9ff1-1fd9ab58b83a None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:42.738874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4867c-3559-4642-9ff1-1fd9ab58b83a None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:42.739366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4867c-3559-4642-9ff1-1fd9ab58b83a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:42.755553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4867c-3559-4642-9ff1-1fd9ab58b83a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:42.758413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4867c-3559-4642-9ff1-1fd9ab58b83a None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-01a4867c-3559-4642-9ff1-1fd9ab58b83a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:42.783413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:42.784416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:42.785386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:42.785386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:42.785386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:42.785386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:42.785386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:42.785386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:42.785386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:42.785386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:42.785386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 08:25:42.785386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:42.786441 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 443/866] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 08:25:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:42.794577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58f00d-d69e-4c3f-9248-ebeb039c829f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:42.935443 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58f00d-d69e-4c3f-9248-ebeb039c829f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:42.978325 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a562a-fd46-4d8e-9f3e-71207c632161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:42.978948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a562a-fd46-4d8e-9f3e-71207c632161 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:42.979224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a562a-fd46-4d8e-9f3e-71207c632161 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:42.979496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a562a-fd46-4d8e-9f3e-71207c632161 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:42.980237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a562a-fd46-4d8e-9f3e-71207c632161 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:42.980540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a562a-fd46-4d8e-9f3e-71207c632161 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:42.982040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a562a-fd46-4d8e-9f3e-71207c632161 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:43.014876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 424/867] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 08:25:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:43.021004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4867c-3559-4642-9ff1-1fd9ab58b83a None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:43.078190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d64e89-c697-45c3-be2d-a6b5753aa6e7 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:43.078551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d64e89-c697-45c3-be2d-a6b5753aa6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:43.078834 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d64e89-c697-45c3-be2d-a6b5753aa6e7 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:43.080922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 425/868] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:43.087514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9490496-54c8-4afb-bd52-ca9896454c60 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:43.087844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9490496-54c8-4afb-bd52-ca9896454c60 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:43.088276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9490496-54c8-4afb-bd52-ca9896454c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:43.114268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-d9490496-54c8-4afb-bd52-ca9896454c60 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:43.115489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d9490496-54c8-4afb-bd52-ca9896454c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:43.118927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:43.166473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4867c-3559-4642-9ff1-1fd9ab58b83a None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:43.206839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 426/869] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:25:43.208218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925793e1-a40b-4564-b161-0f67f87f85f0 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:43.208503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925793e1-a40b-4564-b161-0f67f87f85f0 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:43.208776 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925793e1-a40b-4564-b161-0f67f87f85f0 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:43.214831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 427/870] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:43.216783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7ef01-d40f-4dc1-b54e-5e46ba5db5b9 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:43.217085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7ef01-d40f-4dc1-b54e-5e46ba5db5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:43.217448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7ef01-d40f-4dc1-b54e-5e46ba5db5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:43.228040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31871e8-0e6c-4ffc-8ac7-6ce02f09ec74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:43.228652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31871e8-0e6c-4ffc-8ac7-6ce02f09ec74 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:43.228929 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31871e8-0e6c-4ffc-8ac7-6ce02f09ec74 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:43.229261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31871e8-0e6c-4ffc-8ac7-6ce02f09ec74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:43.255474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ead7ef01-d40f-4dc1-b54e-5e46ba5db5b9 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:43.257390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ead7ef01-d40f-4dc1-b54e-5e46ba5db5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:43.301361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:43.325487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31871e8-0e6c-4ffc-8ac7-6ce02f09ec74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:43.327316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31871e8-0e6c-4ffc-8ac7-6ce02f09ec74 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:43.349117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 428/871] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:25:43.489197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7ef01-d40f-4dc1-b54e-5e46ba5db5b9 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:43.516302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 444/872] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:43 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:25:43.530281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31871e8-0e6c-4ffc-8ac7-6ce02f09ec74 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:43.570806 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:43.610157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7ef01-d40f-4dc1-b54e-5e46ba5db5b9 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:43.626123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:43.647294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31871e8-0e6c-4ffc-8ac7-6ce02f09ec74 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:43.648271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d2753-773b-4137-8583-e3a3e647adb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:43.648980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d2753-773b-4137-8583-e3a3e647adb2 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:43.649309 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d2753-773b-4137-8583-e3a3e647adb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:43.649675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d2753-773b-4137-8583-e3a3e647adb2 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:43.650344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d2753-773b-4137-8583-e3a3e647adb2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:43.651951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d2753-773b-4137-8583-e3a3e647adb2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:43.673136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 429/873] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:25:43 2022] POST /identity/v3/services => generated 269 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:25:43.673136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d2753-773b-4137-8583-e3a3e647adb2 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:43.698325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78bac7-a83e-4076-9c99-ec21b898d3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:43.699700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78bac7-a83e-4076-9c99-ec21b898d3ca None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:43.700179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78bac7-a83e-4076-9c99-ec21b898d3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:43.700623 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78bac7-a83e-4076-9c99-ec21b898d3ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:43.714738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78bac7-a83e-4076-9c99-ec21b898d3ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:43.716463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78bac7-a83e-4076-9c99-ec21b898d3ca None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-3a78bac7-a83e-4076-9c99-ec21b898d3ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:43.736174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:43.737410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:43.738498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:43.738498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:43.738498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:43.740634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 445/874] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:43.746400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78bac7-a83e-4076-9c99-ec21b898d3ca None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:43.816113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d2753-773b-4137-8583-e3a3e647adb2 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:43.857767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:43.868653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1ca72-5da8-4473-9e37-280cd50cbfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:43.869602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1ca72-5da8-4473-9e37-280cd50cbfbc None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:43.870094 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1ca72-5da8-4473-9e37-280cd50cbfbc None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:43.870589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1ca72-5da8-4473-9e37-280cd50cbfbc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:43.871189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1ca72-5da8-4473-9e37-280cd50cbfbc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:43.872566 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1ca72-5da8-4473-9e37-280cd50cbfbc None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:43.877366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78bac7-a83e-4076-9c99-ec21b898d3ca None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-d2a1ca72-5da8-4473-9e37-280cd50cbfbc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:43.881853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:43.882857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:25:43.883765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:43.884496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 430/875] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 08:25:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:43.893650 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1ca72-5da8-4473-9e37-280cd50cbfbc None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:43.917840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:43.931167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8a649-c889-4428-b7d7-b4797d3d7cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:43.932228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8a649-c889-4428-b7d7-b4797d3d7cba None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:43.932646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8a649-c889-4428-b7d7-b4797d3d7cba None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:43.933026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8a649-c889-4428-b7d7-b4797d3d7cba None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:43.933852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8a649-c889-4428-b7d7-b4797d3d7cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:43.934374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8a649-c889-4428-b7d7-b4797d3d7cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:43.936547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8a649-c889-4428-b7d7-b4797d3d7cba None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:43.955252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 446/876] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:43.963390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8a649-c889-4428-b7d7-b4797d3d7cba None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:44.035925 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1ca72-5da8-4473-9e37-280cd50cbfbc None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:44.087634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348c4f6-fbbb-4532-a389-180093d05034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:44.088295 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348c4f6-fbbb-4532-a389-180093d05034 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:44.088568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348c4f6-fbbb-4532-a389-180093d05034 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:44.088829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348c4f6-fbbb-4532-a389-180093d05034 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:44.089486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348c4f6-fbbb-4532-a389-180093d05034 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:44.089782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348c4f6-fbbb-4532-a389-180093d05034 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:44.091165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8348c4f6-fbbb-4532-a389-180093d05034 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:44.097481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 431/877] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 08:25:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:44.102928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348c4f6-fbbb-4532-a389-180093d05034 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:44.114316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8a649-c889-4428-b7d7-b4797d3d7cba None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:44.155252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8d1b6-a8f4-4513-b0a7-22718ebb3be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:44.156732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8d1b6-a8f4-4513-b0a7-22718ebb3be0 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:44.157278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8d1b6-a8f4-4513-b0a7-22718ebb3be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:44.157753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8d1b6-a8f4-4513-b0a7-22718ebb3be0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:44.166367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8d1b6-a8f4-4513-b0a7-22718ebb3be0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:44.168105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8d1b6-a8f4-4513-b0a7-22718ebb3be0 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:44.172913 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-09a8d1b6-a8f4-4513-b0a7-22718ebb3be0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:44.185329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:44.186279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:44.187206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:44.187206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:44.187206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:44.187206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:44.187206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:44.187206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:44.187206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:44.187206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:44.190150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 447/878] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:44.197415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8d1b6-a8f4-4513-b0a7-22718ebb3be0 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:44.225281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348c4f6-fbbb-4532-a389-180093d05034 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:44.258353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:44.269780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72feed9d-3d69-4f98-8ba6-055ac679ac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:44.270539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72feed9d-3d69-4f98-8ba6-055ac679ac7e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:44.270949 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72feed9d-3d69-4f98-8ba6-055ac679ac7e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:44.271262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72feed9d-3d69-4f98-8ba6-055ac679ac7e None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:44.271987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72feed9d-3d69-4f98-8ba6-055ac679ac7e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:44.272386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72feed9d-3d69-4f98-8ba6-055ac679ac7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:44.274072 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72feed9d-3d69-4f98-8ba6-055ac679ac7e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:44.285583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 432/879] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 08:25:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:44.322766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8d1b6-a8f4-4513-b0a7-22718ebb3be0 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:44.363752 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a754484-462d-4544-ac80-2c0b9fff4697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:44.364703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a754484-462d-4544-ac80-2c0b9fff4697 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:44.365185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a754484-462d-4544-ac80-2c0b9fff4697 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:44.365642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a754484-462d-4544-ac80-2c0b9fff4697 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:44.366517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a754484-462d-4544-ac80-2c0b9fff4697 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:44.366979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a754484-462d-4544-ac80-2c0b9fff4697 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:44.368626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a754484-462d-4544-ac80-2c0b9fff4697 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:44.383885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 448/880] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:44.390006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72feed9d-3d69-4f98-8ba6-055ac679ac7e None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:44.513992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72feed9d-3d69-4f98-8ba6-055ac679ac7e None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:44.550113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0003b-5176-4dea-96bb-38a345d63e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:44.550798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0003b-5176-4dea-96bb-38a345d63e4b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:44.551504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:44.552203 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0003b-5176-4dea-96bb-38a345d63e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:44.552660 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0003b-5176-4dea-96bb-38a345d63e4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:44.553597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0003b-5176-4dea-96bb-38a345d63e4b None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=2dae32845ef94e95a9d8f7c16c73c787) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:44.554080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0003b-5176-4dea-96bb-38a345d63e4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:44.556014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0003b-5176-4dea-96bb-38a345d63e4b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:44.569223 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 433/881] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:44 2022] GET /identity/v3/role_assignments?user.id=2dae32845ef94e95a9d8f7c16c73c787&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:44.580855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c32cae-07c8-440e-9d44-78efe08b28e3 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:44.581047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c32cae-07c8-440e-9d44-78efe08b28e3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:44.581358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c32cae-07c8-440e-9d44-78efe08b28e3 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:44.582972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 449/882] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:44.591422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248d6db-f329-4116-b931-0eb5b1080fab None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:44.592075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248d6db-f329-4116-b931-0eb5b1080fab None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:44.592563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248d6db-f329-4116-b931-0eb5b1080fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:44.613341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-3248d6db-f329-4116-b931-0eb5b1080fab None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:44.614980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3248d6db-f329-4116-b931-0eb5b1080fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:44.701022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 434/883] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:44.985508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c32cae-07c8-440e-9d44-78efe08b28e3 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:45.097512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c32cae-07c8-440e-9d44-78efe08b28e3 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:45.109772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:45.134480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc10e2c-b405-4540-8acb-19d2c0115dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:45.135124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc10e2c-b405-4540-8acb-19d2c0115dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:45.135408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc10e2c-b405-4540-8acb-19d2c0115dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:45.135720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc10e2c-b405-4540-8acb-19d2c0115dd4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:45.136309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc10e2c-b405-4540-8acb-19d2c0115dd4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:45.137697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc10e2c-b405-4540-8acb-19d2c0115dd4 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-2bc10e2c-b405-4540-8acb-19d2c0115dd4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:45.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:45.148152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:45.149172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 450/884] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:25:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:45.154679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248d6db-f329-4116-b931-0eb5b1080fab None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:45.271465 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248d6db-f329-4116-b931-0eb5b1080fab None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:45.291651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:45.311281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de292c-d57d-4e90-87e8-3399abdd069c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:45.312846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de292c-d57d-4e90-87e8-3399abdd069c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:45.313327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de292c-d57d-4e90-87e8-3399abdd069c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:45.313796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de292c-d57d-4e90-87e8-3399abdd069c None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:45.314672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de292c-d57d-4e90-87e8-3399abdd069c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:45.315160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de292c-d57d-4e90-87e8-3399abdd069c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:45.316700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de292c-d57d-4e90-87e8-3399abdd069c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:45.322466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 435/885] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:25:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:45.332242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc10e2c-b405-4540-8acb-19d2c0115dd4 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:45.441926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc10e2c-b405-4540-8acb-19d2c0115dd4 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:45.478430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebc967-1477-4d0f-ae7a-bf8050f8870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:45.479045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebc967-1477-4d0f-ae7a-bf8050f8870e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:45.479322 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebc967-1477-4d0f-ae7a-bf8050f8870e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:45.479589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebc967-1477-4d0f-ae7a-bf8050f8870e None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:45.480216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebc967-1477-4d0f-ae7a-bf8050f8870e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:45.480516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebc967-1477-4d0f-ae7a-bf8050f8870e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:45.481980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebc967-1477-4d0f-ae7a-bf8050f8870e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:45.488005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 451/886] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:25:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:45.493576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de292c-d57d-4e90-87e8-3399abdd069c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:45.610425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de292c-d57d-4e90-87e8-3399abdd069c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:45.648130 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03338e49-fda4-459b-89d2-f0d9d152b8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:45.648741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03338e49-fda4-459b-89d2-f0d9d152b8dc None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:45.649014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03338e49-fda4-459b-89d2-f0d9d152b8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:45.649304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03338e49-fda4-459b-89d2-f0d9d152b8dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:45.650000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03338e49-fda4-459b-89d2-f0d9d152b8dc None admin] RBAC: Adding query filter params (interface=public, service_id=ada5b2dff3404055a7713af2fb47d7b3, region_id=RegionOne) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:45.650301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03338e49-fda4-459b-89d2-f0d9d152b8dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:45.651905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03338e49-fda4-459b-89d2-f0d9d152b8dc None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:45.658546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 436/887] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 08:25:45 2022] GET /identity/v3/endpoints?service_id=ada5b2dff3404055a7713af2fb47d7b3&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:45.664186 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebc967-1477-4d0f-ae7a-bf8050f8870e None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:45.706590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:45.770527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebc967-1477-4d0f-ae7a-bf8050f8870e None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:45.807285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423aba54-5f70-45b0-903c-a7f271bbdecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:45.808240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423aba54-5f70-45b0-903c-a7f271bbdecb None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:45.808749 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423aba54-5f70-45b0-903c-a7f271bbdecb None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:45.809254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423aba54-5f70-45b0-903c-a7f271bbdecb None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:45.810055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423aba54-5f70-45b0-903c-a7f271bbdecb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:45.811907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423aba54-5f70-45b0-903c-a7f271bbdecb None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:45.819436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 452/888] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 08:25:45 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:25:46.056344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c5b0d-2fda-41cf-a958-05ae7c77dda2 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:46.056691 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c5b0d-2fda-41cf-a958-05ae7c77dda2 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:46.056885 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c5b0d-2fda-41cf-a958-05ae7c77dda2 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:46.058595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 437/889] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:46.065043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b67380-90a5-4bfb-b7e2-2f4652402b69 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:46.065555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b67380-90a5-4bfb-b7e2-2f4652402b69 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:46.065634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b67380-90a5-4bfb-b7e2-2f4652402b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:46.083826 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-04b67380-90a5-4bfb-b7e2-2f4652402b69 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:46.085016 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-04b67380-90a5-4bfb-b7e2-2f4652402b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:46.158923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 453/890] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:46.446830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c5b0d-2fda-41cf-a958-05ae7c77dda2 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:46.509347 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:46.601914 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c5b0d-2fda-41cf-a958-05ae7c77dda2 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:46.607152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f31cb-b33a-465f-95cf-3e94b12a7bb9 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:46.607727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f31cb-b33a-465f-95cf-3e94b12a7bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:46.608203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f31cb-b33a-465f-95cf-3e94b12a7bb9 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:46.610111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 454/891] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:46.615949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ce330-e0de-4af5-95bd-094361bb1e5a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:46.616484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ce330-e0de-4af5-95bd-094361bb1e5a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:46.616937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ce330-e0de-4af5-95bd-094361bb1e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:46.636815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-6f9ce330-e0de-4af5-95bd-094361bb1e5a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:46.638698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-6f9ce330-e0de-4af5-95bd-094361bb1e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:46.646901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e1c91-2a4a-407b-92c0-a79e2b3daad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:46.647593 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e1c91-2a4a-407b-92c0-a79e2b3daad9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:46.647871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e1c91-2a4a-407b-92c0-a79e2b3daad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:46.648142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e1c91-2a4a-407b-92c0-a79e2b3daad9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:46.648731 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e1c91-2a4a-407b-92c0-a79e2b3daad9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:46.650157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e1c91-2a4a-407b-92c0-a79e2b3daad9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-c79e1c91-2a4a-407b-92c0-a79e2b3daad9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:46.661162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:46.662401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 08:25:46.663306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:46.667218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 438/892] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Sep 28 08:25:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:46.668611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e1c91-2a4a-407b-92c0-a79e2b3daad9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:46.689553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:46.741354 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 455/893] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:25:46.797247 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e1c91-2a4a-407b-92c0-a79e2b3daad9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:46.807179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:46.836224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759c524-6181-4b2f-a1fa-42a99b97e655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:46.836904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759c524-6181-4b2f-a1fa-42a99b97e655 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:46.837212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759c524-6181-4b2f-a1fa-42a99b97e655 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:46.837484 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759c524-6181-4b2f-a1fa-42a99b97e655 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:46.838115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759c524-6181-4b2f-a1fa-42a99b97e655 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:46.838409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759c524-6181-4b2f-a1fa-42a99b97e655 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:46.839782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759c524-6181-4b2f-a1fa-42a99b97e655 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:46.850327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 439/894] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 08:25:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:46.851943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ce330-e0de-4af5-95bd-094361bb1e5a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:46.969028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ce330-e0de-4af5-95bd-094361bb1e5a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:47.005719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6af9e-99dd-43f4-b1fe-8362836995c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:47.006257 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6af9e-99dd-43f4-b1fe-8362836995c3 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:47.006536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6af9e-99dd-43f4-b1fe-8362836995c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:47.006807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6af9e-99dd-43f4-b1fe-8362836995c3 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:47.007450 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6af9e-99dd-43f4-b1fe-8362836995c3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:47.007754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6af9e-99dd-43f4-b1fe-8362836995c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:47.009139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6af9e-99dd-43f4-b1fe-8362836995c3 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:47.014977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 456/895] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 08:25:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:47.020873 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759c524-6181-4b2f-a1fa-42a99b97e655 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:47.031411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68e720-5383-493f-945e-161d2a51543a None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:47.031768 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68e720-5383-493f-945e-161d2a51543a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:47.032028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68e720-5383-493f-945e-161d2a51543a None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:47.033766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 457/896] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:25:47.040848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d262986-fded-45ee-bb0a-52404f34f90d None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:47.041166 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d262986-fded-45ee-bb0a-52404f34f90d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:47.041423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d262986-fded-45ee-bb0a-52404f34f90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:47.062529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-7d262986-fded-45ee-bb0a-52404f34f90d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:47.063744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7d262986-fded-45ee-bb0a-52404f34f90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:47.088361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:47.144631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 458/897] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:47.150973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d262986-fded-45ee-bb0a-52404f34f90d None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:47.153967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759c524-6181-4b2f-a1fa-42a99b97e655 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:47.193672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdaf06-b6e8-4ed1-b127-1ee955b39657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:47.194610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdaf06-b6e8-4ed1-b127-1ee955b39657 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:47.195078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdaf06-b6e8-4ed1-b127-1ee955b39657 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:47.195533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdaf06-b6e8-4ed1-b127-1ee955b39657 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:47.196348 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defdaf06-b6e8-4ed1-b127-1ee955b39657 None admin] RBAC: Adding query filter params (interface=public, service_id=2f05ea8117744e8e9c1f6da31d9bfc1c, region_id=RegionOne) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:47.196756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defdaf06-b6e8-4ed1-b127-1ee955b39657 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:47.198291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defdaf06-b6e8-4ed1-b127-1ee955b39657 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:47.209002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 440/898] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 08:25:47 2022] GET /identity/v3/endpoints?service_id=2f05ea8117744e8e9c1f6da31d9bfc1c&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:47.210308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdaf06-b6e8-4ed1-b127-1ee955b39657 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:47.254379 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:47.294242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d262986-fded-45ee-bb0a-52404f34f90d None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:47.341079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9305c9-cada-48ed-af21-7c4a6c4432ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:47.342024 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9305c9-cada-48ed-af21-7c4a6c4432ba None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:47.344348 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdaf06-b6e8-4ed1-b127-1ee955b39657 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:47.344760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9305c9-cada-48ed-af21-7c4a6c4432ba None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:47.345409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9305c9-cada-48ed-af21-7c4a6c4432ba None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:47.347216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9305c9-cada-48ed-af21-7c4a6c4432ba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:47.348793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9305c9-cada-48ed-af21-7c4a6c4432ba None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:47.373180 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 459/899] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:25:47 2022] POST /identity/v3/services => generated 241 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:25:47.409483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f11d26-6e50-4ab3-83b2-f2a944a330d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:47.410515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f11d26-6e50-4ab3-83b2-f2a944a330d0 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:47.410998 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f11d26-6e50-4ab3-83b2-f2a944a330d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:47.411564 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f11d26-6e50-4ab3-83b2-f2a944a330d0 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:47.412545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f11d26-6e50-4ab3-83b2-f2a944a330d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:47.414352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f11d26-6e50-4ab3-83b2-f2a944a330d0 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:47.428989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 441/900] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 08:25:47 2022] GET /identity/v3/services => generated 1497 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:25:47.430377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9305c9-cada-48ed-af21-7c4a6c4432ba None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:47.454542 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:47.583306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9305c9-cada-48ed-af21-7c4a6c4432ba None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:47.628650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7029e-070e-47fc-83bc-181208fcc33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:47.629340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7029e-070e-47fc-83bc-181208fcc33c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:47.629624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7029e-070e-47fc-83bc-181208fcc33c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:47.629903 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7029e-070e-47fc-83bc-181208fcc33c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:47.642842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7029e-070e-47fc-83bc-181208fcc33c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-dfb7029e-070e-47fc-83bc-181208fcc33c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:47.648366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:47.649340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:47.650357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 460/901] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 08:25:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:47.655864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f11d26-6e50-4ab3-83b2-f2a944a330d0 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:47.804046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:47.815976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f11d26-6e50-4ab3-83b2-f2a944a330d0 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:47.862215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c4c5a-b15d-4809-888e-60132c8be1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:47.862739 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c4c5a-b15d-4809-888e-60132c8be1cc None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:47.863019 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c4c5a-b15d-4809-888e-60132c8be1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:47.863291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c4c5a-b15d-4809-888e-60132c8be1cc None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:47.863960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c4c5a-b15d-4809-888e-60132c8be1cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:47.864261 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c4c5a-b15d-4809-888e-60132c8be1cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:47.865666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c4c5a-b15d-4809-888e-60132c8be1cc None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:47.875434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 442/902] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 08:25:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:47.881292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7029e-070e-47fc-83bc-181208fcc33c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:47.970178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:48.000652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7029e-070e-47fc-83bc-181208fcc33c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:48.039660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ab08d-a62b-44d0-8496-1341d22aa56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:48.040294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ab08d-a62b-44d0-8496-1341d22aa56f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:48.040570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ab08d-a62b-44d0-8496-1341d22aa56f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:48.040839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ab08d-a62b-44d0-8496-1341d22aa56f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:48.052501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ab08d-a62b-44d0-8496-1341d22aa56f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:48.053989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ab08d-a62b-44d0-8496-1341d22aa56f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-5e1ab08d-a62b-44d0-8496-1341d22aa56f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:48.070697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:48.071766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:48.072716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:48.072716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:48.072716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:48.072716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 461/903] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:48.078715 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c4c5a-b15d-4809-888e-60132c8be1cc None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:48.107605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebed512-382d-4974-8869-bf72a6eb2c02 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:48.108168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebed512-382d-4974-8869-bf72a6eb2c02 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:48.108247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebed512-382d-4974-8869-bf72a6eb2c02 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:48.110355 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 462/904] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:48.118160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747ffff-fbdb-4650-9acb-4f51a158f136 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:48.118759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747ffff-fbdb-4650-9acb-4f51a158f136 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:48.119271 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747ffff-fbdb-4650-9acb-4f51a158f136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:48.143772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e747ffff-fbdb-4650-9acb-4f51a158f136 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:48.145410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e747ffff-fbdb-4650-9acb-4f51a158f136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:48.195451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c4c5a-b15d-4809-888e-60132c8be1cc None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:48.237505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 463/905] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:25:48.242866 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ea11f-a887-471a-b02f-bce3610b0966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:48.243677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ea11f-a887-471a-b02f-bce3610b0966 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:48.244122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ea11f-a887-471a-b02f-bce3610b0966 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:48.244541 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ea11f-a887-471a-b02f-bce3610b0966 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:48.245435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ea11f-a887-471a-b02f-bce3610b0966 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:48.245912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ea11f-a887-471a-b02f-bce3610b0966 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:48.247487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ea11f-a887-471a-b02f-bce3610b0966 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:48.261986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 443/906] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:48.268636 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747ffff-fbdb-4650-9acb-4f51a158f136 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:48.337879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:48.388510 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747ffff-fbdb-4650-9acb-4f51a158f136 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:48.442453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac591b98-2ce3-48b3-91f4-9f5969f63f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:48.443576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac591b98-2ce3-48b3-91f4-9f5969f63f44 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:48.444072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac591b98-2ce3-48b3-91f4-9f5969f63f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:48.444569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac591b98-2ce3-48b3-91f4-9f5969f63f44 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:48.457072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac591b98-2ce3-48b3-91f4-9f5969f63f44 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:48.458788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac591b98-2ce3-48b3-91f4-9f5969f63f44 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-ac591b98-2ce3-48b3-91f4-9f5969f63f44 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:48.472990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:48.474010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:48.474979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:48.474979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:48.474979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:48.474979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:48.474979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:48.474979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:48.474979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:48.474979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:48.474979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 08:25:48.474979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:48.474979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 464/907] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 08:25:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:48.482556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ea11f-a887-471a-b02f-bce3610b0966 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:48.504681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:48.544008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac591b98-2ce3-48b3-91f4-9f5969f63f44 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:48.628881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ea11f-a887-471a-b02f-bce3610b0966 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:48.667338 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac591b98-2ce3-48b3-91f4-9f5969f63f44 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:48.698095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557544b-b355-4088-9eaf-c6be1c25fa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:48.700195 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557544b-b355-4088-9eaf-c6be1c25fa5a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:48.701005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557544b-b355-4088-9eaf-c6be1c25fa5a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:48.701853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557544b-b355-4088-9eaf-c6be1c25fa5a None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:48.704114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557544b-b355-4088-9eaf-c6be1c25fa5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:48.705147 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557544b-b355-4088-9eaf-c6be1c25fa5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:48.708846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557544b-b355-4088-9eaf-c6be1c25fa5a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:48.715753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9c4be-94d0-4314-a104-08d6cb66d78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:48.716686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9c4be-94d0-4314-a104-08d6cb66d78e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:48.717204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9c4be-94d0-4314-a104-08d6cb66d78e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:48.717690 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9c4be-94d0-4314-a104-08d6cb66d78e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:48.718970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9c4be-94d0-4314-a104-08d6cb66d78e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:48.719711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:48.731514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9c4be-94d0-4314-a104-08d6cb66d78e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-30c9c4be-94d0-4314-a104-08d6cb66d78e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:48.742079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:48.743963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:48.745733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 465/908] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:25:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:48.748804 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9c4be-94d0-4314-a104-08d6cb66d78e None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:48.761269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 444/909] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 08:25:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:48.766625 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557544b-b355-4088-9eaf-c6be1c25fa5a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:48.865522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9c4be-94d0-4314-a104-08d6cb66d78e None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:48.869663 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:48.882407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557544b-b355-4088-9eaf-c6be1c25fa5a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:48.910771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90149c-bf59-49fe-a41a-3b2d8c383776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:48.911518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90149c-bf59-49fe-a41a-3b2d8c383776 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:48.911720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90149c-bf59-49fe-a41a-3b2d8c383776 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:48.912002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90149c-bf59-49fe-a41a-3b2d8c383776 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:48.912646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90149c-bf59-49fe-a41a-3b2d8c383776 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:48.912954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90149c-bf59-49fe-a41a-3b2d8c383776 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:48.914384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90149c-bf59-49fe-a41a-3b2d8c383776 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:48.925438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 466/910] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:25:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:48.926491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90149c-bf59-49fe-a41a-3b2d8c383776 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:48.939149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba7bd9-e597-48d5-b720-1cdd71700a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:48.940173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba7bd9-e597-48d5-b720-1cdd71700a4e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:48.940671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba7bd9-e597-48d5-b720-1cdd71700a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:48.941204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba7bd9-e597-48d5-b720-1cdd71700a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:49.023809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:49.045912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba7bd9-e597-48d5-b720-1cdd71700a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:49.048583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba7bd9-e597-48d5-b720-1cdd71700a4e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:49.114035 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90149c-bf59-49fe-a41a-3b2d8c383776 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:49.170854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:49.173046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d3523-21ab-48a6-a255-c53bb441f8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:49.174005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d3523-21ab-48a6-a255-c53bb441f8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:49.174952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d3523-21ab-48a6-a255-c53bb441f8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:49.175630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d3523-21ab-48a6-a255-c53bb441f8e0 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:49.176801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d3523-21ab-48a6-a255-c53bb441f8e0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:49.177556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d3523-21ab-48a6-a255-c53bb441f8e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:49.179763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d3523-21ab-48a6-a255-c53bb441f8e0 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:49.191888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 467/911] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 08:25:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:49.197548 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d3523-21ab-48a6-a255-c53bb441f8e0 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:49.217679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 445/912] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:48 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:25:49.219448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba7bd9-e597-48d5-b720-1cdd71700a4e None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:49.332297 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d3523-21ab-48a6-a255-c53bb441f8e0 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:49.341870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba7bd9-e597-48d5-b720-1cdd71700a4e None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:49.346339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:49.378358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe4f01-b5f1-4809-8a05-7d0bedbff55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:49.379398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe4f01-b5f1-4809-8a05-7d0bedbff55b None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:49.379885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe4f01-b5f1-4809-8a05-7d0bedbff55b None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:49.380407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe4f01-b5f1-4809-8a05-7d0bedbff55b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:49.381366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe4f01-b5f1-4809-8a05-7d0bedbff55b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:49.383325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe4f01-b5f1-4809-8a05-7d0bedbff55b None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:49.392440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da9809-e08b-474c-9a01-4a1c275675d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:49.393108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da9809-e08b-474c-9a01-4a1c275675d2 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:49.393424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da9809-e08b-474c-9a01-4a1c275675d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:49.393989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da9809-e08b-474c-9a01-4a1c275675d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:49.408337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da9809-e08b-474c-9a01-4a1c275675d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:49.409885 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da9809-e08b-474c-9a01-4a1c275675d2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:49.425948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 468/913] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:25:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-d7da9809-e08b-474c-9a01-4a1c275675d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:49.427075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:49.428024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:49.428901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:49.428901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:49.428901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:49.428901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 446/914] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:49.434435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe4f01-b5f1-4809-8a05-7d0bedbff55b None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:49.573037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe4f01-b5f1-4809-8a05-7d0bedbff55b None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:49.615870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e0c2b-912f-40c9-a04f-1d0ed98c881f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:49.616497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e0c2b-912f-40c9-a04f-1d0ed98c881f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:49.616827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e0c2b-912f-40c9-a04f-1d0ed98c881f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:49.617210 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e0c2b-912f-40c9-a04f-1d0ed98c881f None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:49.617884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e0c2b-912f-40c9-a04f-1d0ed98c881f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:49.618188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e0c2b-912f-40c9-a04f-1d0ed98c881f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:49.619593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e0c2b-912f-40c9-a04f-1d0ed98c881f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:49.635459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 469/915] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:49.641813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da9809-e08b-474c-9a01-4a1c275675d2 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:49.680266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:49.773081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da9809-e08b-474c-9a01-4a1c275675d2 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:49.821690 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48f608-af8e-4b7b-b9cc-b8d9688e74fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:49.822784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48f608-af8e-4b7b-b9cc-b8d9688e74fe None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:49.823266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48f608-af8e-4b7b-b9cc-b8d9688e74fe None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:49.823742 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48f608-af8e-4b7b-b9cc-b8d9688e74fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:49.832793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48f608-af8e-4b7b-b9cc-b8d9688e74fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:49.834632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48f608-af8e-4b7b-b9cc-b8d9688e74fe None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:49.837146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aeb6c-3f26-4b67-952f-336cd168c36e None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:49.837436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aeb6c-3f26-4b67-952f-336cd168c36e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:49.837710 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aeb6c-3f26-4b67-952f-336cd168c36e None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:49.839310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 470/916] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:49.845515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d462fb-f2a0-44af-bdd8-3d91389d8ebc None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:49.845801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d462fb-f2a0-44af-bdd8-3d91389d8ebc None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:49.846063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d462fb-f2a0-44af-bdd8-3d91389d8ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-aa48f608-af8e-4b7b-b9cc-b8d9688e74fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:49.850159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:49.851047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:49.851918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:49.851918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:49.851918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:49.851918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:49.851918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:49.851918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:49.851918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:49.851918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:49.852390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 447/917] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:49.858245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48f608-af8e-4b7b-b9cc-b8d9688e74fe None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:49.871979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-c6d462fb-f2a0-44af-bdd8-3d91389d8ebc None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:49.874448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c6d462fb-f2a0-44af-bdd8-3d91389d8ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:49.898453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:49.969452 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:49.983627 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48f608-af8e-4b7b-b9cc-b8d9688e74fe None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:49.993661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 471/918] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:25:49.993854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f85bf-d1f2-469b-b918-e9204a5b7346 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:49.994147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f85bf-d1f2-469b-b918-e9204a5b7346 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:49.994489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f85bf-d1f2-469b-b918-e9204a5b7346 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:49.996351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 472/919] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:50.003495 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2ad6e-9ab4-4557-bb1a-64297eb6009b None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:50.003779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2ad6e-9ab4-4557-bb1a-64297eb6009b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:50.004064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2ad6e-9ab4-4557-bb1a-64297eb6009b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:50.024530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-d7a2ad6e-9ab4-4557-bb1a-64297eb6009b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:50.025206 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291d654-ef1a-4c0b-a8cd-759712ba92ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:50.025862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d7a2ad6e-9ab4-4557-bb1a-64297eb6009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:50.027718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291d654-ef1a-4c0b-a8cd-759712ba92ff None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:50.028197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291d654-ef1a-4c0b-a8cd-759712ba92ff None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:50.028516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291d654-ef1a-4c0b-a8cd-759712ba92ff None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:50.029253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291d654-ef1a-4c0b-a8cd-759712ba92ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:50.029559 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291d654-ef1a-4c0b-a8cd-759712ba92ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:50.031059 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291d654-ef1a-4c0b-a8cd-759712ba92ff None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:50.045130 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 448/920] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:50.051574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291d654-ef1a-4c0b-a8cd-759712ba92ff None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:50.130014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 473/921] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:50.175388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291d654-ef1a-4c0b-a8cd-759712ba92ff None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:50.213862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd835a2-4c32-4eff-9e82-8c4b2e713881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:50.214473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd835a2-4c32-4eff-9e82-8c4b2e713881 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:50.214809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd835a2-4c32-4eff-9e82-8c4b2e713881 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:50.215127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd835a2-4c32-4eff-9e82-8c4b2e713881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:50.215862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd835a2-4c32-4eff-9e82-8c4b2e713881 None admin] RBAC: Adding query filter params (role.id=323909a5cc584ec1957e40e8928f214f, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=2dae32845ef94e95a9d8f7c16c73c787) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:50.216172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd835a2-4c32-4eff-9e82-8c4b2e713881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:50.217703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd835a2-4c32-4eff-9e82-8c4b2e713881 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:50.224055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 449/922] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:25:50 2022] GET /identity/v3/role_assignments?user.id=2dae32845ef94e95a9d8f7c16c73c787&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=323909a5cc584ec1957e40e8928f214f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:50.287836 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2ad6e-9ab4-4557-bb1a-64297eb6009b None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:50.383290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:50.403553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2ad6e-9ab4-4557-bb1a-64297eb6009b None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:50.427357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd835a2-4c32-4eff-9e82-8c4b2e713881 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:50.440350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708cfea-5ad3-4ec6-b430-760ec8eeb1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:50.440986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708cfea-5ad3-4ec6-b430-760ec8eeb1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:50.441294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708cfea-5ad3-4ec6-b430-760ec8eeb1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:50.441569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708cfea-5ad3-4ec6-b430-760ec8eeb1d9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:50.442154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708cfea-5ad3-4ec6-b430-760ec8eeb1d9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:50.444000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708cfea-5ad3-4ec6-b430-760ec8eeb1d9 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:50.451568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-d708cfea-5ad3-4ec6-b430-760ec8eeb1d9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:50.459976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:50.461015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 08:25:50.461945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:50.464100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 474/923] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Sep 28 08:25:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:50.469184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708cfea-5ad3-4ec6-b430-760ec8eeb1d9 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:50.574193 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd835a2-4c32-4eff-9e82-8c4b2e713881 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:50.593571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708cfea-5ad3-4ec6-b430-760ec8eeb1d9 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:50.614447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fc4ba-93dd-4753-bddc-07f4ef3a3fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:50.615189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fc4ba-93dd-4753-bddc-07f4ef3a3fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:50.615564 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fc4ba-93dd-4753-bddc-07f4ef3a3fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:50.615935 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fc4ba-93dd-4753-bddc-07f4ef3a3fb4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:50.616655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fc4ba-93dd-4753-bddc-07f4ef3a3fb4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:50.618237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fc4ba-93dd-4753-bddc-07f4ef3a3fb4 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-181fc4ba-93dd-4753-bddc-07f4ef3a3fb4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:50.631797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:50.633144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:25:50.634211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:50.641146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 450/924] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 08:25:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:50.643795 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fc4ba-93dd-4753-bddc-07f4ef3a3fb4 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:50.645720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa32f33-8420-4600-83d8-ee6481d55d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:50.646599 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa32f33-8420-4600-83d8-ee6481d55d8c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:50.647432 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa32f33-8420-4600-83d8-ee6481d55d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:50.647922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa32f33-8420-4600-83d8-ee6481d55d8c None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:50.649809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa32f33-8420-4600-83d8-ee6481d55d8c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:50.650332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa32f33-8420-4600-83d8-ee6481d55d8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:50.652156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa32f33-8420-4600-83d8-ee6481d55d8c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:50.660306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 475/925] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 08:25:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:50.666696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa32f33-8420-4600-83d8-ee6481d55d8c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:50.681587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:50.739378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:50.766747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fc4ba-93dd-4753-bddc-07f4ef3a3fb4 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:50.819743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f20fb6-9eb5-4a7c-8fe7-962f4adc6b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:50.820368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f20fb6-9eb5-4a7c-8fe7-962f4adc6b0b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:50.820616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f20fb6-9eb5-4a7c-8fe7-962f4adc6b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:50.820797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f20fb6-9eb5-4a7c-8fe7-962f4adc6b0b None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:50.821442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f20fb6-9eb5-4a7c-8fe7-962f4adc6b0b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:50.821759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f20fb6-9eb5-4a7c-8fe7-962f4adc6b0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:50.823129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f20fb6-9eb5-4a7c-8fe7-962f4adc6b0b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:50.824633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa32f33-8420-4600-83d8-ee6481d55d8c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:50.833622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 451/926] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 08:25:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:50.838025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f20fb6-9eb5-4a7c-8fe7-962f4adc6b0b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:50.871609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a9963-c4f8-469f-a62b-4abf037d9e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:50.872223 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a9963-c4f8-469f-a62b-4abf037d9e59 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:50.872552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a9963-c4f8-469f-a62b-4abf037d9e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:50.872973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a9963-c4f8-469f-a62b-4abf037d9e59 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:50.873670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a9963-c4f8-469f-a62b-4abf037d9e59 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:50.873973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a9963-c4f8-469f-a62b-4abf037d9e59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:50.876185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a9963-c4f8-469f-a62b-4abf037d9e59 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:50.882881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 476/927] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 08:25:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:50.888092 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a9963-c4f8-469f-a62b-4abf037d9e59 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:50.975835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f20fb6-9eb5-4a7c-8fe7-962f4adc6b0b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:50.982190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:51.013528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a9963-c4f8-469f-a62b-4abf037d9e59 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:51.021135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd41570-4a78-4b17-bf65-7078d9ac875c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:51.021934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd41570-4a78-4b17-bf65-7078d9ac875c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:51.022319 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd41570-4a78-4b17-bf65-7078d9ac875c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:51.022765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd41570-4a78-4b17-bf65-7078d9ac875c None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:51.023497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd41570-4a78-4b17-bf65-7078d9ac875c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:51.023932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd41570-4a78-4b17-bf65-7078d9ac875c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:51.025424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd41570-4a78-4b17-bf65-7078d9ac875c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:51.037003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 452/928] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 08:25:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:51.038320 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd41570-4a78-4b17-bf65-7078d9ac875c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:51.042123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:51.051229 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b62e9e-e9a8-4594-a3cf-c06637d45228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:51.051845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b62e9e-e9a8-4594-a3cf-c06637d45228 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:51.052127 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b62e9e-e9a8-4594-a3cf-c06637d45228 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:51.052400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b62e9e-e9a8-4594-a3cf-c06637d45228 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:51.053231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b62e9e-e9a8-4594-a3cf-c06637d45228 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:51.054703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b62e9e-e9a8-4594-a3cf-c06637d45228 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:51.085654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 477/929] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:25:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:25:51.175509 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd41570-4a78-4b17-bf65-7078d9ac875c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:51.225955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed107b3-d971-499e-8036-7d2baa2ea736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:51.226996 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed107b3-d971-499e-8036-7d2baa2ea736 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:51.227532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed107b3-d971-499e-8036-7d2baa2ea736 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:51.228026 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed107b3-d971-499e-8036-7d2baa2ea736 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:51.228962 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed107b3-d971-499e-8036-7d2baa2ea736 None admin] RBAC: Adding query filter params (interface=public, service_id=afe8b5fa584f4f8dbd96bb12894b465b, region_id=RegionOne) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:51.229518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed107b3-d971-499e-8036-7d2baa2ea736 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:51.231190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed107b3-d971-499e-8036-7d2baa2ea736 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:51.241238 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 453/930] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 08:25:51 2022] GET /identity/v3/endpoints?service_id=afe8b5fa584f4f8dbd96bb12894b465b&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:51.246272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b62e9e-e9a8-4594-a3cf-c06637d45228 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:51.357878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b62e9e-e9a8-4594-a3cf-c06637d45228 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:51.388185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:51.396140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83212155-19a8-40a8-99f2-38864c461e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:51.396760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83212155-19a8-40a8-99f2-38864c461e69 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:51.397055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83212155-19a8-40a8-99f2-38864c461e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:51.397351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83212155-19a8-40a8-99f2-38864c461e69 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:51.398156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83212155-19a8-40a8-99f2-38864c461e69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:51.399705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83212155-19a8-40a8-99f2-38864c461e69 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:51.411562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 478/931] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 08:25:51 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:25:51.825681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7ca5a-1084-4575-86ae-3c06d9a3089d None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:51.826231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7ca5a-1084-4575-86ae-3c06d9a3089d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:51.826231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7ca5a-1084-4575-86ae-3c06d9a3089d None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:51.828317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 454/932] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:51.833736 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db27ae-8e78-4ad1-ac39-0e1a2b66409e None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:51.834019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db27ae-8e78-4ad1-ac39-0e1a2b66409e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:51.834294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db27ae-8e78-4ad1-ac39-0e1a2b66409e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:51.853373 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-00db27ae-8e78-4ad1-ac39-0e1a2b66409e None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:51.854633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-00db27ae-8e78-4ad1-ac39-0e1a2b66409e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:51.927490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 479/933] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:52.215633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7ca5a-1084-4575-86ae-3c06d9a3089d None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:52.283879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:52.329264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7ca5a-1084-4575-86ae-3c06d9a3089d None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:52.372066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce69fce-7de1-413b-aa77-97a1a70a8367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:52.373178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce69fce-7de1-413b-aa77-97a1a70a8367 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:52.373674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce69fce-7de1-413b-aa77-97a1a70a8367 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:52.374171 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce69fce-7de1-413b-aa77-97a1a70a8367 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:52.375156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce69fce-7de1-413b-aa77-97a1a70a8367 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:52.376939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce69fce-7de1-413b-aa77-97a1a70a8367 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:52.380067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: WARNING keystone.server.flask.common [None req-5ce69fce-7de1-413b-aa77-97a1a70a8367 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 28 08:25:52.401611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.sql.core [None req-5ce69fce-7de1-413b-aa77-97a1a70a8367 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 08:25:52.401611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 28 08:25:52.401611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [parameters: {'id': '714b8c180cdd4fb8b5ca2bd616db3c2b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 28 08:25:52.401611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81483) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-5ce69fce-7de1-413b-aa77-97a1a70a8367 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 714b8c180cdd4fb8b5ca2bd616db3c2b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 714b8c180cdd4fb8b5ca2bd616db3c2b. Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application conn.query(q) Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 08:25:52.410649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application result.read() Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 28 08:25:52.411435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application next(self.gen) Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application yield resource Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application next(self.gen) Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise self.value Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application session.commit() Sep 28 08:25:52.412439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self.session.flush() Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application compat.raise_( Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 08:25:52.413376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 08:25:52.415022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application conn.query(q) Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application result.read() Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 08:25:52.416369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [parameters: {'id': '714b8c180cdd4fb8b5ca2bd616db3c2b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:52.418619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:52.420536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:52.423286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:52.423286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 28 08:25:52.423286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 28 08:25:52.423286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:52.423286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:52.423286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 28 08:25:52.423286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 28 08:25:52.423286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 714b8c180cdd4fb8b5ca2bd616db3c2b. Sep 28 08:25:52.423286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:52.426280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 455/934] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Sep 28 08:25:52 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:52.437241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db27ae-8e78-4ad1-ac39-0e1a2b66409e None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:52.486935 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e145a-0a00-4277-b925-cf1b7ed2c857 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:52.487507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e145a-0a00-4277-b925-cf1b7ed2c857 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:52.488270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e145a-0a00-4277-b925-cf1b7ed2c857 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:52.494433 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 456/935] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:52.499741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c1d74-7c6b-4b20-8101-02ce994cc8e6 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:52.500124 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c1d74-7c6b-4b20-8101-02ce994cc8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:52.500400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c1d74-7c6b-4b20-8101-02ce994cc8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:52.525642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-e69c1d74-7c6b-4b20-8101-02ce994cc8e6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:52.528035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e69c1d74-7c6b-4b20-8101-02ce994cc8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:52.556040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db27ae-8e78-4ad1-ac39-0e1a2b66409e None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:52.583512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:52.598374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc86d5-dda7-4a73-aa85-5ecbc7644fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:52.599465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc86d5-dda7-4a73-aa85-5ecbc7644fca None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:52.600014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc86d5-dda7-4a73-aa85-5ecbc7644fca None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:52.600481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc86d5-dda7-4a73-aa85-5ecbc7644fca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:52.608309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc86d5-dda7-4a73-aa85-5ecbc7644fca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:52.610833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc86d5-dda7-4a73-aa85-5ecbc7644fca None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:52.619099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 457/936] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-85dc86d5-dda7-4a73-aa85-5ecbc7644fca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:52.623168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:52.624263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:52.625753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:52.625753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:52.625753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:52.625753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:52.625753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:52.625753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:52.625753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:52.625753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:52.629185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 480/937] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:52.637883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c1d74-7c6b-4b20-8101-02ce994cc8e6 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:52.694846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:52.750834 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c1d74-7c6b-4b20-8101-02ce994cc8e6 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:52.786844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad62e41-de13-4f88-8785-b008f8bb60b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:52.787453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad62e41-de13-4f88-8785-b008f8bb60b0 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:52.787733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad62e41-de13-4f88-8785-b008f8bb60b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:52.788005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad62e41-de13-4f88-8785-b008f8bb60b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:52.788669 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad62e41-de13-4f88-8785-b008f8bb60b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:52.788970 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad62e41-de13-4f88-8785-b008f8bb60b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:52.790554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad62e41-de13-4f88-8785-b008f8bb60b0 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:52.809451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 458/938] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:25:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:52.895310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc86d5-dda7-4a73-aa85-5ecbc7644fca None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:53.010884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc86d5-dda7-4a73-aa85-5ecbc7644fca None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:53.047498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e128d7c-3876-4119-8c35-4f68849db8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:53.048132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e128d7c-3876-4119-8c35-4f68849db8ff None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:53.048406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e128d7c-3876-4119-8c35-4f68849db8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:53.048675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e128d7c-3876-4119-8c35-4f68849db8ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:53.059488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e128d7c-3876-4119-8c35-4f68849db8ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-3e128d7c-3876-4119-8c35-4f68849db8ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:53.065273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:53.066203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:53.067033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 481/939] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 08:25:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:53.072344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad62e41-de13-4f88-8785-b008f8bb60b0 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:53.176801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad62e41-de13-4f88-8785-b008f8bb60b0 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:53.188493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:53.213705 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28366ca-3fb8-48a9-a105-f77b84a677fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:53.214363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28366ca-3fb8-48a9-a105-f77b84a677fd None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:53.214641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28366ca-3fb8-48a9-a105-f77b84a677fd None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:53.214908 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28366ca-3fb8-48a9-a105-f77b84a677fd None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:53.215570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28366ca-3fb8-48a9-a105-f77b84a677fd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:53.215868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28366ca-3fb8-48a9-a105-f77b84a677fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:53.217248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28366ca-3fb8-48a9-a105-f77b84a677fd None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:53.226320 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 459/940] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 08:25:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:53.231829 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e128d7c-3876-4119-8c35-4f68849db8ff None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:53.290802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:53.348458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e128d7c-3876-4119-8c35-4f68849db8ff None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:53.386629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2191db3-0761-4659-93a7-e7a958a35d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:53.387285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2191db3-0761-4659-93a7-e7a958a35d9d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:53.387561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2191db3-0761-4659-93a7-e7a958a35d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:53.387828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2191db3-0761-4659-93a7-e7a958a35d9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:53.401127 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2191db3-0761-4659-93a7-e7a958a35d9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:53.402853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2191db3-0761-4659-93a7-e7a958a35d9d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-e2191db3-0761-4659-93a7-e7a958a35d9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:53.418889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:53.419893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:53.420792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:53.420792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:53.420792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:53.420792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 482/941] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:53.425969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28366ca-3fb8-48a9-a105-f77b84a677fd None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:53.503334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93334-2c46-44b5-93eb-215f44e904a3 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:53.504304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93334-2c46-44b5-93eb-215f44e904a3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:53.504828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93334-2c46-44b5-93eb-215f44e904a3 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:53.506687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 483/942] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:25:53.512095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fa3bf-cc53-4f52-9c6c-328b029f1fd8 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:53.512632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fa3bf-cc53-4f52-9c6c-328b029f1fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:53.518716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fa3bf-cc53-4f52-9c6c-328b029f1fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:53.535577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28366ca-3fb8-48a9-a105-f77b84a677fd None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:53.543310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-6c6fa3bf-cc53-4f52-9c6c-328b029f1fd8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:53.545160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-6c6fa3bf-cc53-4f52-9c6c-328b029f1fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:53.579608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b7b4b-4dd8-4135-8a41-c3ca773c94b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:53.580391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b7b4b-4dd8-4135-8a41-c3ca773c94b1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:53.580772 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b7b4b-4dd8-4135-8a41-c3ca773c94b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:53.581185 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b7b4b-4dd8-4135-8a41-c3ca773c94b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:53.582038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b7b4b-4dd8-4135-8a41-c3ca773c94b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:53.582554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b7b4b-4dd8-4135-8a41-c3ca773c94b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:53.584139 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b7b4b-4dd8-4135-8a41-c3ca773c94b1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:53.598684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 460/943] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:53.604453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b7b4b-4dd8-4135-8a41-c3ca773c94b1 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:53.633485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 484/944] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:53.647659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:53.734721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b7b4b-4dd8-4135-8a41-c3ca773c94b1 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:53.777639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89ae28-f111-47da-bafd-a27155279e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:53.778580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89ae28-f111-47da-bafd-a27155279e36 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:53.779042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89ae28-f111-47da-bafd-a27155279e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:53.779493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89ae28-f111-47da-bafd-a27155279e36 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:53.787438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89ae28-f111-47da-bafd-a27155279e36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:53.789147 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89ae28-f111-47da-bafd-a27155279e36 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:53.791853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efded86-6f42-456e-8de4-979b6d954398 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:53.792143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efded86-6f42-456e-8de4-979b6d954398 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:53.792427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efded86-6f42-456e-8de4-979b6d954398 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:53.795137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 485/945] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:25:53.799204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a1a2-cae5-48e8-a388-fdd30a04ac9d None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:53.799490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a1a2-cae5-48e8-a388-fdd30a04ac9d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:53.799763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a1a2-cae5-48e8-a388-fdd30a04ac9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-ad89ae28-f111-47da-bafd-a27155279e36 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:53.803446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:53.804402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:53.805384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:53.805384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:53.805384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:53.805384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:53.805384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:53.805384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:53.805384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:53.805384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:53.805384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 08:25:53.805384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:53.807552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 461/946] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 08:25:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:53.813677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89ae28-f111-47da-bafd-a27155279e36 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:53.822550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-cdf4a1a2-cae5-48e8-a388-fdd30a04ac9d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:53.824368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-cdf4a1a2-cae5-48e8-a388-fdd30a04ac9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:53.857520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:53.917419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 486/947] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:25:53.928109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a1a2-cae5-48e8-a388-fdd30a04ac9d None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:53.937493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:53.950175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89ae28-f111-47da-bafd-a27155279e36 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:54.000067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be6104-0f1c-480e-bb2a-5ce0fe900dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:54.001348 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be6104-0f1c-480e-bb2a-5ce0fe900dcf None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:54.001965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be6104-0f1c-480e-bb2a-5ce0fe900dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:54.002449 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be6104-0f1c-480e-bb2a-5ce0fe900dcf None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:54.003629 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be6104-0f1c-480e-bb2a-5ce0fe900dcf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:54.004734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be6104-0f1c-480e-bb2a-5ce0fe900dcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:54.006661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be6104-0f1c-480e-bb2a-5ce0fe900dcf None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:54.034969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 462/948] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 08:25:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:54.039101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be6104-0f1c-480e-bb2a-5ce0fe900dcf None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:54.061802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a1a2-cae5-48e8-a388-fdd30a04ac9d None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:54.104093 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7daa8b-cef9-4765-9c5b-96e1f7183885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:54.105213 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7daa8b-cef9-4765-9c5b-96e1f7183885 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:54.105900 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7daa8b-cef9-4765-9c5b-96e1f7183885 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:54.106497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7daa8b-cef9-4765-9c5b-96e1f7183885 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:54.107341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7daa8b-cef9-4765-9c5b-96e1f7183885 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:54.109820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7daa8b-cef9-4765-9c5b-96e1f7183885 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-9c7daa8b-cef9-4765-9c5b-96e1f7183885 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:54.120121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:54.121200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 08:25:54.122029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:54.130392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 487/949] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:54.131586 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7daa8b-cef9-4765-9c5b-96e1f7183885 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:54.157054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:54.157977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be6104-0f1c-480e-bb2a-5ce0fe900dcf None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:54.197203 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1599c9-14c6-4bbe-9172-e396802320e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:54.197846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1599c9-14c6-4bbe-9172-e396802320e4 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:54.198126 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1599c9-14c6-4bbe-9172-e396802320e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:54.198393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1599c9-14c6-4bbe-9172-e396802320e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:54.210190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1599c9-14c6-4bbe-9172-e396802320e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:54.211642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1599c9-14c6-4bbe-9172-e396802320e4 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-6c1599c9-14c6-4bbe-9172-e396802320e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:54.228803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:54.229971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:54.230848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:54.230848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:54.230848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:54.230848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 463/950] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:54.230848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1599c9-14c6-4bbe-9172-e396802320e4 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:54.233628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:54.250052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7daa8b-cef9-4765-9c5b-96e1f7183885 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:54.284359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45830697-e937-4c6e-abad-529d9ebe72d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:54.284970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830697-e937-4c6e-abad-529d9ebe72d8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:54.285291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830697-e937-4c6e-abad-529d9ebe72d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:54.285563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830697-e937-4c6e-abad-529d9ebe72d8 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:54.286181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45830697-e937-4c6e-abad-529d9ebe72d8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:54.286501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45830697-e937-4c6e-abad-529d9ebe72d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:54.287822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45830697-e937-4c6e-abad-529d9ebe72d8 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:54.293134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 488/951] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:25:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:54.293395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45830697-e937-4c6e-abad-529d9ebe72d8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:54.348995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1599c9-14c6-4bbe-9172-e396802320e4 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:54.384594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72b878-27ff-4228-804d-33d3bb733b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:54.385280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72b878-27ff-4228-804d-33d3bb733b99 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:54.385558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72b878-27ff-4228-804d-33d3bb733b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:54.385830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df72b878-27ff-4228-804d-33d3bb733b99 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:54.386416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72b878-27ff-4228-804d-33d3bb733b99 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:54.387924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df72b878-27ff-4228-804d-33d3bb733b99 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:54.399642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-df72b878-27ff-4228-804d-33d3bb733b99 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:54.402774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:54.403844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:54.404953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 464/952] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 08:25:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:54.410184 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72b878-27ff-4228-804d-33d3bb733b99 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:54.425257 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45830697-e937-4c6e-abad-529d9ebe72d8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:54.461818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eef9b8-30fc-499a-8bc9-8b0dae5f7bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:54.462878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eef9b8-30fc-499a-8bc9-8b0dae5f7bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:54.463162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eef9b8-30fc-499a-8bc9-8b0dae5f7bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:54.463591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eef9b8-30fc-499a-8bc9-8b0dae5f7bc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:54.464242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eef9b8-30fc-499a-8bc9-8b0dae5f7bc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:54.464549 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eef9b8-30fc-499a-8bc9-8b0dae5f7bc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:54.465930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eef9b8-30fc-499a-8bc9-8b0dae5f7bc8 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:54.475411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:54.479530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 489/953] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:54.480520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eef9b8-30fc-499a-8bc9-8b0dae5f7bc8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:54.518409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df72b878-27ff-4228-804d-33d3bb733b99 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:54.557870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429154bf-cb2d-4020-8bb3-2a099428a663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:54.558638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429154bf-cb2d-4020-8bb3-2a099428a663 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:54.560033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429154bf-cb2d-4020-8bb3-2a099428a663 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:54.560033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429154bf-cb2d-4020-8bb3-2a099428a663 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:54.560033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429154bf-cb2d-4020-8bb3-2a099428a663 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:54.560444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429154bf-cb2d-4020-8bb3-2a099428a663 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:54.561472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429154bf-cb2d-4020-8bb3-2a099428a663 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:54.574459 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 465/954] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:25:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:54.574459 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429154bf-cb2d-4020-8bb3-2a099428a663 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:54.606655 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eef9b8-30fc-499a-8bc9-8b0dae5f7bc8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:54.660408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec283db-920c-417f-9a53-f77323bcbc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:54.661209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec283db-920c-417f-9a53-f77323bcbc1a None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:54.661597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec283db-920c-417f-9a53-f77323bcbc1a None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:54.661989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec283db-920c-417f-9a53-f77323bcbc1a None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:54.663048 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec283db-920c-417f-9a53-f77323bcbc1a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:54.663494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec283db-920c-417f-9a53-f77323bcbc1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:54.665754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec283db-920c-417f-9a53-f77323bcbc1a None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:54.673170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 490/955] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 08:25:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:54.686603 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec283db-920c-417f-9a53-f77323bcbc1a None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:54.699854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:54.708580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429154bf-cb2d-4020-8bb3-2a099428a663 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:54.743515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:54.751912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48aff7-b81c-4953-8bd6-5d89bb07bc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:54.752560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48aff7-b81c-4953-8bd6-5d89bb07bc0c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:54.752896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48aff7-b81c-4953-8bd6-5d89bb07bc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:54.753524 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48aff7-b81c-4953-8bd6-5d89bb07bc0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:54.761136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48aff7-b81c-4953-8bd6-5d89bb07bc0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:54.762680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48aff7-b81c-4953-8bd6-5d89bb07bc0c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-ea48aff7-b81c-4953-8bd6-5d89bb07bc0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:54.775771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:54.777079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:54.778076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:54.778076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:54.778076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:54.778076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:54.778076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:54.778076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:54.778076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:54.778076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:54.782770 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 466/956] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:54.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48aff7-b81c-4953-8bd6-5d89bb07bc0c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:54.818523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec283db-920c-417f-9a53-f77323bcbc1a None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:54.880114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0ae11-6225-41ac-b547-d578fcc8e032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:54.881018 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0ae11-6225-41ac-b547-d578fcc8e032 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:54.881418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0ae11-6225-41ac-b547-d578fcc8e032 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:54.881727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0ae11-6225-41ac-b547-d578fcc8e032 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:54.882832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0ae11-6225-41ac-b547-d578fcc8e032 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:54.883192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0ae11-6225-41ac-b547-d578fcc8e032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:54.885146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0ae11-6225-41ac-b547-d578fcc8e032 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:54.893497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 491/957] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 08:25:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:54.902782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0ae11-6225-41ac-b547-d578fcc8e032 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:54.928803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48aff7-b81c-4953-8bd6-5d89bb07bc0c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:54.970447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4cc73-0abd-4e6b-ad45-51e6a6545cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:54.971220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4cc73-0abd-4e6b-ad45-51e6a6545cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:54.971603 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4cc73-0abd-4e6b-ad45-51e6a6545cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:54.972001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4cc73-0abd-4e6b-ad45-51e6a6545cd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:54.972796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4cc73-0abd-4e6b-ad45-51e6a6545cd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:54.973231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4cc73-0abd-4e6b-ad45-51e6a6545cd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:54.974763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4cc73-0abd-4e6b-ad45-51e6a6545cd0 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:54.981478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:54.988662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 467/958] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:54.997029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4cc73-0abd-4e6b-ad45-51e6a6545cd0 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:55.025363 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0ae11-6225-41ac-b547-d578fcc8e032 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:55.053792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:55.065606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8610ac0-b2c2-4cab-927a-0acec0184968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:55.066711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8610ac0-b2c2-4cab-927a-0acec0184968 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:55.067255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8610ac0-b2c2-4cab-927a-0acec0184968 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:55.067734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8610ac0-b2c2-4cab-927a-0acec0184968 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:55.068665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8610ac0-b2c2-4cab-927a-0acec0184968 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:55.070331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8610ac0-b2c2-4cab-927a-0acec0184968 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:55.105332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 492/959] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:25:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:25:55.116407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4cc73-0abd-4e6b-ad45-51e6a6545cd0 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:55.165147 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15466349-04cd-4370-9c41-90f317e29707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:55.165919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15466349-04cd-4370-9c41-90f317e29707 None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:55.166085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15466349-04cd-4370-9c41-90f317e29707 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:55.166351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15466349-04cd-4370-9c41-90f317e29707 None admin] PATH_INFO: `/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/2dae32845ef94e95a9d8f7c16c73c787/roles/323909a5cc584ec1957e40e8928f214f` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:55.168744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15466349-04cd-4370-9c41-90f317e29707 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca4d4801f2f40419f6881984fbbfbcd, user_id=2dae32845ef94e95a9d8f7c16c73c787, role_id=323909a5cc584ec1957e40e8928f214f)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:55.170395 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15466349-04cd-4370-9c41-90f317e29707 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:55.189746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a097bd1-51a1-40ad-90b5-8da7b1cfd4bd None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:55.190062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a097bd1-51a1-40ad-90b5-8da7b1cfd4bd None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:55.190313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 468/960] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:25:54 2022] PUT /identity/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/2dae32845ef94e95a9d8f7c16c73c787/roles/323909a5cc584ec1957e40e8928f214f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:25:55.190408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a097bd1-51a1-40ad-90b5-8da7b1cfd4bd None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:55.191835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 493/961] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:25:55.197517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fbcc4-2959-4f5a-8dc0-5b62198a01ba None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:55.197869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fbcc4-2959-4f5a-8dc0-5b62198a01ba None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:55.198076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fbcc4-2959-4f5a-8dc0-5b62198a01ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:55.223845 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-b51fbcc4-2959-4f5a-8dc0-5b62198a01ba None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:55.225234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b51fbcc4-2959-4f5a-8dc0-5b62198a01ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:55.308672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 469/962] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:55.593697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a097bd1-51a1-40ad-90b5-8da7b1cfd4bd None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:55.704841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a097bd1-51a1-40ad-90b5-8da7b1cfd4bd None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:55.731874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:55.740618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bbbcc-1155-4b7d-b848-e6ab81ec3d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:55.741276 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bbbcc-1155-4b7d-b848-e6ab81ec3d2a None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:55.741966 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bbbcc-1155-4b7d-b848-e6ab81ec3d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:55.742249 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bbbcc-1155-4b7d-b848-e6ab81ec3d2a None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:55.742911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bbbcc-1155-4b7d-b848-e6ab81ec3d2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:55.744454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bbbcc-1155-4b7d-b848-e6ab81ec3d2a None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:55.747514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: WARNING keystone.server.flask.common [None req-770bbbcc-1155-4b7d-b848-e6ab81ec3d2a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 28 08:25:55.762036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.sql.core [None req-770bbbcc-1155-4b7d-b848-e6ab81ec3d2a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 08:25:55.762036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 28 08:25:55.762036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [parameters: {'user_id': 'cb2f93babeb24fd4bc2c4c9f651fdbb0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 28 08:25:55.762036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81482) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-770bbbcc-1155-4b7d-b848-e6ab81ec3d2a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application conn.query(q) Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application result.read() Sep 28 08:25:55.779834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 08:25:55.780795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application next(self.gen) Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application yield resource Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application next(self.gen) Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise self.value Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application session.commit() Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 08:25:55.782268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self.session.flush() Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application compat.raise_( Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 08:25:55.783146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 08:25:55.784040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application conn.query(q) Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application result.read() Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 08:25:55.784919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cb2f93babeb24fd4bc2c4c9f651fdbb0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:55.787605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 28 08:25:55.789573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 28 08:25:55.800181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 28 08:25:55.800181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 28 08:25:55.800181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 08:25:55.800181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 08:25:55.800181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 28 08:25:55.800181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:55.800181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 494/963] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 08:25:55 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:55.801159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fbcc4-2959-4f5a-8dc0-5b62198a01ba None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:55.837301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:55.913532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fbcc4-2959-4f5a-8dc0-5b62198a01ba None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:55.951651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f18f2a-4e1c-4767-87c7-29adbdfd2ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:55.953311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f18f2a-4e1c-4767-87c7-29adbdfd2ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:55.953806 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f18f2a-4e1c-4767-87c7-29adbdfd2ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:55.954317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f18f2a-4e1c-4767-87c7-29adbdfd2ac1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:55.962538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f18f2a-4e1c-4767-87c7-29adbdfd2ac1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:55.964300 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f18f2a-4e1c-4767-87c7-29adbdfd2ac1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-30f18f2a-4e1c-4767-87c7-29adbdfd2ac1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:55.976971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:55.978005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:55.978937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:55.978937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:55.978937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:55.978937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:55.978937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:55.978937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:55.978937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:55.978937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:55.978937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 08:25:55.978937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:55.981558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 470/964] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:25:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:55.987871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bbbcc-1155-4b7d-b848-e6ab81ec3d2a None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:56.104007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bbbcc-1155-4b7d-b848-e6ab81ec3d2a None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:56.139834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c20171-69fd-449d-a6d4-e72b9c2d3e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:56.141047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c20171-69fd-449d-a6d4-e72b9c2d3e6e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:56.141356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c20171-69fd-449d-a6d4-e72b9c2d3e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:56.141632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c20171-69fd-449d-a6d4-e72b9c2d3e6e None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:56.142334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c20171-69fd-449d-a6d4-e72b9c2d3e6e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:56.142638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c20171-69fd-449d-a6d4-e72b9c2d3e6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:56.144266 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c20171-69fd-449d-a6d4-e72b9c2d3e6e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:56.185438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 495/965] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 08:25:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:56.924933 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f37573-a575-4d4e-a599-c50245027ece None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:56.925464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f37573-a575-4d4e-a599-c50245027ece None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:56.925464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f37573-a575-4d4e-a599-c50245027ece None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:56.926817 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 471/966] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:56.932257 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872980b-4e66-403d-b0a3-57890e842d95 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:56.932683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872980b-4e66-403d-b0a3-57890e842d95 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:56.932852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872980b-4e66-403d-b0a3-57890e842d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:56.951589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f872980b-4e66-403d-b0a3-57890e842d95 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:56.953543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f872980b-4e66-403d-b0a3-57890e842d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:57.006925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:57.034693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 496/967] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:25:57.303675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f37573-a575-4d4e-a599-c50245027ece None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:57.375216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebb2c6-d640-4db8-8677-340da40162db None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:57.375568 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebb2c6-d640-4db8-8677-340da40162db None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:57.375773 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebb2c6-d640-4db8-8677-340da40162db None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:57.377344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 497/968] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:25:57.383899 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cceec-3a06-4e98-8524-0f8ca70286d6 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:57.384232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cceec-3a06-4e98-8524-0f8ca70286d6 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:57.384457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cceec-3a06-4e98-8524-0f8ca70286d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:57.403454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-948cceec-3a06-4e98-8524-0f8ca70286d6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:57.404625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-948cceec-3a06-4e98-8524-0f8ca70286d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:57.411171 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:57.418543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f37573-a575-4d4e-a599-c50245027ece None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:57.455949 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bc237-bc6e-4311-9ce2-00b49cced1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:57.456558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bc237-bc6e-4311-9ce2-00b49cced1d9 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:57.456831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bc237-bc6e-4311-9ce2-00b49cced1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:57.457123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bc237-bc6e-4311-9ce2-00b49cced1d9 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:57.457777 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bc237-bc6e-4311-9ce2-00b49cced1d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:57.459130 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bc237-bc6e-4311-9ce2-00b49cced1d9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:57.475310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 472/969] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:25:57 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:25:57.475310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c992c83-1373-4a33-a05b-5595ebf0f61b None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:57.477403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c992c83-1373-4a33-a05b-5595ebf0f61b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:57.477658 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c992c83-1373-4a33-a05b-5595ebf0f61b None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:57.479085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 473/970] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:25:57.485362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e20af-4cd3-4f26-bd91-f61e781429a9 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:57.485623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e20af-4cd3-4f26-bd91-f61e781429a9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:57.485903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e20af-4cd3-4f26-bd91-f61e781429a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:57.493844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 498/971] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:25:57.510094 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-395e20af-4cd3-4f26-bd91-f61e781429a9 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:57.511324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-395e20af-4cd3-4f26-bd91-f61e781429a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:57.596523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 474/972] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:25:57.763574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cceec-3a06-4e98-8524-0f8ca70286d6 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:57.863160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e20af-4cd3-4f26-bd91-f61e781429a9 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:57.868644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cceec-3a06-4e98-8524-0f8ca70286d6 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:57.871133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:57.910800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93473a-a47e-408d-addd-8bb5a145d162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:57.911816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93473a-a47e-408d-addd-8bb5a145d162 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:57.912109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93473a-a47e-408d-addd-8bb5a145d162 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:57.912381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93473a-a47e-408d-addd-8bb5a145d162 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:57.913137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93473a-a47e-408d-addd-8bb5a145d162 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:57.914505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93473a-a47e-408d-addd-8bb5a145d162 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-5a93473a-a47e-408d-addd-8bb5a145d162 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:57.924274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:57.925323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:57.926145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 499/973] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 08:25:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:57.931903 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93473a-a47e-408d-addd-8bb5a145d162 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:57.956574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:57.974453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e20af-4cd3-4f26-bd91-f61e781429a9 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:58.015931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef6aa8-153e-416c-89df-f7c9ff57d63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:58.016561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef6aa8-153e-416c-89df-f7c9ff57d63c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:58.016842 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef6aa8-153e-416c-89df-f7c9ff57d63c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:58.017129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef6aa8-153e-416c-89df-f7c9ff57d63c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:58.028334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef6aa8-153e-416c-89df-f7c9ff57d63c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-06ef6aa8-153e-416c-89df-f7c9ff57d63c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:25:58.033793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:58.034891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:58.036274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 475/974] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 08:25:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:58.042041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef6aa8-153e-416c-89df-f7c9ff57d63c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:58.056571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93473a-a47e-408d-addd-8bb5a145d162 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:58.097269 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b39e5f-6d86-49c1-9b2f-62df1345bb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:58.097883 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b39e5f-6d86-49c1-9b2f-62df1345bb95 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:58.098175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b39e5f-6d86-49c1-9b2f-62df1345bb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:58.098439 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b39e5f-6d86-49c1-9b2f-62df1345bb95 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:58.099476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b39e5f-6d86-49c1-9b2f-62df1345bb95 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:58.099801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b39e5f-6d86-49c1-9b2f-62df1345bb95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:58.101217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b39e5f-6d86-49c1-9b2f-62df1345bb95 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:58.106268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 500/975] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 08:25:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:58.112604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b39e5f-6d86-49c1-9b2f-62df1345bb95 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:58.150829 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:58.162929 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef6aa8-153e-416c-89df-f7c9ff57d63c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:58.203121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f41e53-30ce-4644-adea-32ca1543b77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:58.203733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f41e53-30ce-4644-adea-32ca1543b77a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:58.204015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f41e53-30ce-4644-adea-32ca1543b77a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:58.204290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f41e53-30ce-4644-adea-32ca1543b77a None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:58.205371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f41e53-30ce-4644-adea-32ca1543b77a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:58.205683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f41e53-30ce-4644-adea-32ca1543b77a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:58.207684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f41e53-30ce-4644-adea-32ca1543b77a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:58.225321 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 476/976] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 08:25:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:58.227068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f41e53-30ce-4644-adea-32ca1543b77a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:58.228595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b39e5f-6d86-49c1-9b2f-62df1345bb95 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:58.239496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:58.267911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe005c82-1ef4-4b12-8efd-bbbf8464ad74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:58.268778 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe005c82-1ef4-4b12-8efd-bbbf8464ad74 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:58.269057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe005c82-1ef4-4b12-8efd-bbbf8464ad74 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:58.269342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe005c82-1ef4-4b12-8efd-bbbf8464ad74 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:58.269968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe005c82-1ef4-4b12-8efd-bbbf8464ad74 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:58.270270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe005c82-1ef4-4b12-8efd-bbbf8464ad74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:58.271619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe005c82-1ef4-4b12-8efd-bbbf8464ad74 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:58.281502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 501/977] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 08:25:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:58.282927 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe005c82-1ef4-4b12-8efd-bbbf8464ad74 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:58.338706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f41e53-30ce-4644-adea-32ca1543b77a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:58.377565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9705b45-366e-4616-a02f-26b106b4f67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:58.378312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9705b45-366e-4616-a02f-26b106b4f67c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:58.378652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9705b45-366e-4616-a02f-26b106b4f67c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:58.378994 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9705b45-366e-4616-a02f-26b106b4f67c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:58.388653 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:58.396692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9705b45-366e-4616-a02f-26b106b4f67c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:58.398206 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9705b45-366e-4616-a02f-26b106b4f67c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:58.398322 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe005c82-1ef4-4b12-8efd-bbbf8464ad74 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-e9705b45-366e-4616-a02f-26b106b4f67c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:58.413407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:58.414495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:58.415362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:58.415362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:58.415362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:58.415362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 477/978] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:58.420273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9705b45-366e-4616-a02f-26b106b4f67c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:58.440791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b480f-e0e8-4ad2-8340-959f04657483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:58.441438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b480f-e0e8-4ad2-8340-959f04657483 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:58.441720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b480f-e0e8-4ad2-8340-959f04657483 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:58.441990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b480f-e0e8-4ad2-8340-959f04657483 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:58.442690 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b480f-e0e8-4ad2-8340-959f04657483 None admin] RBAC: Adding query filter params (interface=admin, service_id=afe8b5fa584f4f8dbd96bb12894b465b, region_id=RegionOne) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:58.443001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b480f-e0e8-4ad2-8340-959f04657483 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:58.444345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b480f-e0e8-4ad2-8340-959f04657483 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:58.449455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 502/979] 10.222.0.52 () {56 vars in 1249 bytes} [Wed Sep 28 08:25:58 2022] GET /identity/v3/endpoints?service_id=afe8b5fa584f4f8dbd96bb12894b465b&interface=admin®ion_id=RegionOne => generated 193 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:25:58.456002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b480f-e0e8-4ad2-8340-959f04657483 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:58.503422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:58.549614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9705b45-366e-4616-a02f-26b106b4f67c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:58.585836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9e688-d8ec-47cb-a79b-f4be3af40b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:58.586463 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9e688-d8ec-47cb-a79b-f4be3af40b1f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:58.586726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9e688-d8ec-47cb-a79b-f4be3af40b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:58.586993 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9e688-d8ec-47cb-a79b-f4be3af40b1f None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:58.587632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9e688-d8ec-47cb-a79b-f4be3af40b1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:58.587929 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9e688-d8ec-47cb-a79b-f4be3af40b1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:58.588302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b480f-e0e8-4ad2-8340-959f04657483 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:58.589324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9e688-d8ec-47cb-a79b-f4be3af40b1f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:58.602902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 478/980] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:58.608578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9e688-d8ec-47cb-a79b-f4be3af40b1f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:58.628616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3dc3ff-803a-40b4-9027-69a56ef968e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:58.629360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dc3ff-803a-40b4-9027-69a56ef968e2 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:58.629643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dc3ff-803a-40b4-9027-69a56ef968e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:58.629917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dc3ff-803a-40b4-9027-69a56ef968e2 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:58.630648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3dc3ff-803a-40b4-9027-69a56ef968e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:58.632077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3dc3ff-803a-40b4-9027-69a56ef968e2 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:58.649196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 503/981] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 08:25:58 2022] GET /identity/v3/services => generated 1727 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:25:58.721407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9e688-d8ec-47cb-a79b-f4be3af40b1f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:58.744341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:58.762031 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f756882-fee0-4e73-8693-97157323b870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:58.762680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f756882-fee0-4e73-8693-97157323b870 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:58.762958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f756882-fee0-4e73-8693-97157323b870 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:58.763228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f756882-fee0-4e73-8693-97157323b870 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:58.772020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f756882-fee0-4e73-8693-97157323b870 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:58.773539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f756882-fee0-4e73-8693-97157323b870 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-1f756882-fee0-4e73-8693-97157323b870 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:25:58.787158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:58.788175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:58.789120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:58.789120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:58.789120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:25:58.789120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:25:58.789120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:25:58.789120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:25:58.789120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:25:58.789120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:25:58.789120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 08:25:58.789120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:25:58.789120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 479/982] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 08:25:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:25:58.794442 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3dc3ff-803a-40b4-9027-69a56ef968e2 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:58.824030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:58.905654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3dc3ff-803a-40b4-9027-69a56ef968e2 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:58.940625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c27d02-84d7-40f9-8d5f-d7937aaadfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:58.941270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c27d02-84d7-40f9-8d5f-d7937aaadfda None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:58.941570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c27d02-84d7-40f9-8d5f-d7937aaadfda None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:58.941836 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c27d02-84d7-40f9-8d5f-d7937aaadfda None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:58.942557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c27d02-84d7-40f9-8d5f-d7937aaadfda None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:58.942867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c27d02-84d7-40f9-8d5f-d7937aaadfda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:58.944275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c27d02-84d7-40f9-8d5f-d7937aaadfda None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:58.962706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 504/983] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 08:25:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:58.968662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f756882-fee0-4e73-8693-97157323b870 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:59.097446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f756882-fee0-4e73-8693-97157323b870 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:59.136180 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4b76b-63cd-49e6-9833-83f1deb97a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:59.136798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4b76b-63cd-49e6-9833-83f1deb97a83 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:59.137127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4b76b-63cd-49e6-9833-83f1deb97a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:59.137393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4b76b-63cd-49e6-9833-83f1deb97a83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:59.188866 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:59.210035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4b76b-63cd-49e6-9833-83f1deb97a83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:59.211477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d4b76b-63cd-49e6-9833-83f1deb97a83 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:59.317378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 480/984] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:25:58 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:25:59.318990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c27d02-84d7-40f9-8d5f-d7937aaadfda None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:59.404179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:59.424294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c27d02-84d7-40f9-8d5f-d7937aaadfda None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:59.460846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101f2e4-853f-4853-b052-bec3ce369cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:59.461547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101f2e4-853f-4853-b052-bec3ce369cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:59.461832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101f2e4-853f-4853-b052-bec3ce369cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:59.462108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101f2e4-853f-4853-b052-bec3ce369cd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:59.472975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101f2e4-853f-4853-b052-bec3ce369cd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:59.474444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101f2e4-853f-4853-b052-bec3ce369cd1 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-4101f2e4-853f-4853-b052-bec3ce369cd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:25:59.489062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:59.490011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:25:59.490907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:25:59.490907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:25:59.490907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:59.490907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 505/985] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:25:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:59.497064 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4b76b-63cd-49e6-9833-83f1deb97a83 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:59.600900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:59.624108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d4b76b-63cd-49e6-9833-83f1deb97a83 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:59.662791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2603b-b313-4fb4-8976-fb9940423fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:59.663315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2603b-b313-4fb4-8976-fb9940423fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:59.663593 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2603b-b313-4fb4-8976-fb9940423fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:59.663865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2603b-b313-4fb4-8976-fb9940423fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:59.664507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2603b-b313-4fb4-8976-fb9940423fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:25:59.664803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2603b-b313-4fb4-8976-fb9940423fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:59.666199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2603b-b313-4fb4-8976-fb9940423fb3 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:59.681972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 481/986] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:25:59.682984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101f2e4-853f-4853-b052-bec3ce369cd1 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:59.714453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378fbc9-225d-4cd1-a0e4-1aa62920fab8 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:59.714754 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378fbc9-225d-4cd1-a0e4-1aa62920fab8 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:59.715007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378fbc9-225d-4cd1-a0e4-1aa62920fab8 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:59.716597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 482/987] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:25:59.721842 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6795e3-060a-4fbf-95a4-3821ad3de2f4 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:59.722129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6795e3-060a-4fbf-95a4-3821ad3de2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:59.722402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6795e3-060a-4fbf-95a4-3821ad3de2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:59.741923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ae6795e3-060a-4fbf-95a4-3821ad3de2f4 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:25:59.743115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ae6795e3-060a-4fbf-95a4-3821ad3de2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:25:59.789592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101f2e4-853f-4853-b052-bec3ce369cd1 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:25:59.813947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 483/988] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:25:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:25:59.828525 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abed3c-8476-41f7-a045-d57406e44f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:25:59.832893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abed3c-8476-41f7-a045-d57406e44f5e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:25:59.833425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abed3c-8476-41f7-a045-d57406e44f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:25:59.833864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abed3c-8476-41f7-a045-d57406e44f5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:25:59.846181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:59.852974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abed3c-8476-41f7-a045-d57406e44f5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:25:59.854949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abed3c-8476-41f7-a045-d57406e44f5e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-d2abed3c-8476-41f7-a045-d57406e44f5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:25:59.867349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:25:59.868254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:25:59.873846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:25:59.873846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:25:59.873846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:25:59.873846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:25:59.873846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:25:59.873846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:25:59.873846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:25:59.873846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:25:59.873846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 506/989] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:25:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:25:59.876237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6795e3-060a-4fbf-95a4-3821ad3de2f4 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:25:59.948743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:25:59.981775 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6795e3-060a-4fbf-95a4-3821ad3de2f4 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:00.017647 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa81f4f-0292-450d-b56f-63cda0f32a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:00.018278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa81f4f-0292-450d-b56f-63cda0f32a6e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:00.018571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa81f4f-0292-450d-b56f-63cda0f32a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:00.018832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa81f4f-0292-450d-b56f-63cda0f32a6e None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:00.019524 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa81f4f-0292-450d-b56f-63cda0f32a6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:00.019824 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa81f4f-0292-450d-b56f-63cda0f32a6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:00.021270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa81f4f-0292-450d-b56f-63cda0f32a6e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:00.038032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 484/990] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:25:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:00.039641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abed3c-8476-41f7-a045-d57406e44f5e None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:00.081766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa81f4f-0292-450d-b56f-63cda0f32a6e None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:00.149905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abed3c-8476-41f7-a045-d57406e44f5e None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:00.185181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc278e6-f066-44b0-9afe-9d75e894a1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:00.185797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc278e6-f066-44b0-9afe-9d75e894a1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:00.186076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc278e6-f066-44b0-9afe-9d75e894a1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:00.186412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc278e6-f066-44b0-9afe-9d75e894a1b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:00.187616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc278e6-f066-44b0-9afe-9d75e894a1b3 None admin] RBAC: Adding query filter params (role.id=323909a5cc584ec1957e40e8928f214f, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=2dae32845ef94e95a9d8f7c16c73c787) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:00.187926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc278e6-f066-44b0-9afe-9d75e894a1b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:00.189369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc278e6-f066-44b0-9afe-9d75e894a1b3 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:00.195178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 507/991] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:26:00 2022] GET /identity/v3/role_assignments?user.id=2dae32845ef94e95a9d8f7c16c73c787&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=323909a5cc584ec1957e40e8928f214f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:00.216846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa81f4f-0292-450d-b56f-63cda0f32a6e None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:00.251234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:00.258400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca898b7-3717-4a50-bc99-431f468e64e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:00.259064 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca898b7-3717-4a50-bc99-431f468e64e9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:00.259346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca898b7-3717-4a50-bc99-431f468e64e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:00.259617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca898b7-3717-4a50-bc99-431f468e64e9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:00.260211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca898b7-3717-4a50-bc99-431f468e64e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:00.261687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca898b7-3717-4a50-bc99-431f468e64e9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-bca898b7-3717-4a50-bc99-431f468e64e9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:00.271644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:00.272700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:00.278350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 485/992] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:26:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:26:00.279818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc278e6-f066-44b0-9afe-9d75e894a1b3 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:00.345190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:00.398051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc278e6-f066-44b0-9afe-9d75e894a1b3 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:00.434413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8fb13-30d9-4ea7-a0f8-8549f5c86f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:00.435422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8fb13-30d9-4ea7-a0f8-8549f5c86f84 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:00.435924 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8fb13-30d9-4ea7-a0f8-8549f5c86f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:00.437507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8fb13-30d9-4ea7-a0f8-8549f5c86f84 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:00.437507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8fb13-30d9-4ea7-a0f8-8549f5c86f84 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:00.437746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8fb13-30d9-4ea7-a0f8-8549f5c86f84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:00.439153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8fb13-30d9-4ea7-a0f8-8549f5c86f84 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:00.449055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 508/993] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:26:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:00.450581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca898b7-3717-4a50-bc99-431f468e64e9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:00.563973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca898b7-3717-4a50-bc99-431f468e64e9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:00.601018 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59acab9f-786a-4b9f-a3e2-ab0f758a0ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:00.601649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acab9f-786a-4b9f-a3e2-ab0f758a0ddf None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:00.601931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acab9f-786a-4b9f-a3e2-ab0f758a0ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:00.602205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acab9f-786a-4b9f-a3e2-ab0f758a0ddf None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:00.602833 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59acab9f-786a-4b9f-a3e2-ab0f758a0ddf None admin] RBAC: Adding query filter params (type=compute) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:00.603139 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59acab9f-786a-4b9f-a3e2-ab0f758a0ddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:00.604621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59acab9f-786a-4b9f-a3e2-ab0f758a0ddf None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:00.610427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 486/994] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:26:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:00.616288 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8fb13-30d9-4ea7-a0f8-8549f5c86f84 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:00.723581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8fb13-30d9-4ea7-a0f8-8549f5c86f84 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:00.750290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:00.764383 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ec5dd-6b2b-438f-8bea-22da5e20a4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:00.765148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ec5dd-6b2b-438f-8bea-22da5e20a4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:00.765437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ec5dd-6b2b-438f-8bea-22da5e20a4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:00.765712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ec5dd-6b2b-438f-8bea-22da5e20a4a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:00.766431 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ec5dd-6b2b-438f-8bea-22da5e20a4a9 None admin] RBAC: Adding query filter params (interface=public, service_id=9442a64e9cbb475fbd30c60bb24dd01c, region_id=RegionOne) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:00.766746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ec5dd-6b2b-438f-8bea-22da5e20a4a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:00.768119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ec5dd-6b2b-438f-8bea-22da5e20a4a9 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:00.773964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 509/995] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 08:26:00 2022] GET /identity/v3/endpoints?service_id=9442a64e9cbb475fbd30c60bb24dd01c&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:00.781508 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59acab9f-786a-4b9f-a3e2-ab0f758a0ddf None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:00.845750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:00.868245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f817b3-d0cc-428e-a58a-f3f2aa268053 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:00.868579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f817b3-d0cc-428e-a58a-f3f2aa268053 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:00.868815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f817b3-d0cc-428e-a58a-f3f2aa268053 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:00.870302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 510/996] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:00.876263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737afd4a-d2f2-43c5-aa08-583206653143 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:00.876551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737afd4a-d2f2-43c5-aa08-583206653143 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:00.876835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737afd4a-d2f2-43c5-aa08-583206653143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:00.895198 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-737afd4a-d2f2-43c5-aa08-583206653143 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:00.896395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-737afd4a-d2f2-43c5-aa08-583206653143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:00.897217 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59acab9f-786a-4b9f-a3e2-ab0f758a0ddf None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:00.935887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d9810-3a55-49b6-9e44-376dd016135c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:00.936504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d9810-3a55-49b6-9e44-376dd016135c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:00.937450 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d9810-3a55-49b6-9e44-376dd016135c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:00.937744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d9810-3a55-49b6-9e44-376dd016135c None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:00.938355 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d9810-3a55-49b6-9e44-376dd016135c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:00.939723 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d9810-3a55-49b6-9e44-376dd016135c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:00.946251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 487/997] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 08:26:00 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:26:00.972061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 511/998] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:26:01.253999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d9810-3a55-49b6-9e44-376dd016135c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:01.357632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d9810-3a55-49b6-9e44-376dd016135c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:01.381624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:01.394259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b03c11-d090-479a-a483-30048089ac7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:01.394891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b03c11-d090-479a-a483-30048089ac7a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:01.395181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b03c11-d090-479a-a483-30048089ac7a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:01.395437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b03c11-d090-479a-a483-30048089ac7a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:01.396021 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b03c11-d090-479a-a483-30048089ac7a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:01.397400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b03c11-d090-479a-a483-30048089ac7a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-69b03c11-d090-479a-a483-30048089ac7a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:01.407125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:01.408220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:01.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 488/999] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 08:26:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:01.415128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737afd4a-d2f2-43c5-aa08-583206653143 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:01.530108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737afd4a-d2f2-43c5-aa08-583206653143 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:01.548693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:01.567572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951e256-2774-4e0b-9de0-809d1f25ab09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:01.568192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951e256-2774-4e0b-9de0-809d1f25ab09 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:01.568474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951e256-2774-4e0b-9de0-809d1f25ab09 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:01.568747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951e256-2774-4e0b-9de0-809d1f25ab09 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:01.569410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951e256-2774-4e0b-9de0-809d1f25ab09 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:01.569718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951e256-2774-4e0b-9de0-809d1f25ab09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:01.571072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951e256-2774-4e0b-9de0-809d1f25ab09 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:01.576488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 512/1000] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 08:26:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:01.585231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b03c11-d090-479a-a483-30048089ac7a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:01.688977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b03c11-d090-479a-a483-30048089ac7a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:01.724656 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635dcf44-3daf-4bfe-a950-7e494835b3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:01.725301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dcf44-3daf-4bfe-a950-7e494835b3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:01.725580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dcf44-3daf-4bfe-a950-7e494835b3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:01.725851 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dcf44-3daf-4bfe-a950-7e494835b3a7 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:01.726476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dcf44-3daf-4bfe-a950-7e494835b3a7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:01.726774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dcf44-3daf-4bfe-a950-7e494835b3a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:01.728132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635dcf44-3daf-4bfe-a950-7e494835b3a7 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:01.737887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 489/1001] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 08:26:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:01.742070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951e256-2774-4e0b-9de0-809d1f25ab09 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:01.849591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951e256-2774-4e0b-9de0-809d1f25ab09 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:01.884464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119f7c8-db90-401b-b1e0-ae188879882c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:01.885073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119f7c8-db90-401b-b1e0-ae188879882c None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:01.885395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119f7c8-db90-401b-b1e0-ae188879882c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:01.885668 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119f7c8-db90-401b-b1e0-ae188879882c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:01.886328 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119f7c8-db90-401b-b1e0-ae188879882c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:01.887689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119f7c8-db90-401b-b1e0-ae188879882c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:01.905110 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 513/1002] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:26:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:26:02.425895 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b200c-7030-44ad-a483-e3b3ad62fc0b None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:02.426499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b200c-7030-44ad-a483-e3b3ad62fc0b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:02.426499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b200c-7030-44ad-a483-e3b3ad62fc0b None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:02.427926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 490/1003] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:02.433062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fe832-963a-4d22-a90c-0d382d253b42 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:02.433535 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fe832-963a-4d22-a90c-0d382d253b42 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:02.433626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fe832-963a-4d22-a90c-0d382d253b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:02.452829 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-955fe832-963a-4d22-a90c-0d382d253b42 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:02.454214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-955fe832-963a-4d22-a90c-0d382d253b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:02.490696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:02.538514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 514/1004] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:02.801881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b200c-7030-44ad-a483-e3b3ad62fc0b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:02.857361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:02.914219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b200c-7030-44ad-a483-e3b3ad62fc0b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:02.948741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b5b1e-5452-486a-a369-0547e6bf0a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:02.949401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b5b1e-5452-486a-a369-0547e6bf0a7f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:02.949682 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b5b1e-5452-486a-a369-0547e6bf0a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:02.949950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b5b1e-5452-486a-a369-0547e6bf0a7f None admin] PATH_INFO: `/v3/services/image` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:02.950521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b5b1e-5452-486a-a369-0547e6bf0a7f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:02.951891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b5b1e-5452-486a-a369-0547e6bf0a7f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-178b5b1e-5452-486a-a369-0547e6bf0a7f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:02.960872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:02.962068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 08:26:02.963123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:02.964582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 491/1005] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Sep 28 08:26:02 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:02.967867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fe832-963a-4d22-a90c-0d382d253b42 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:03.083227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fe832-963a-4d22-a90c-0d382d253b42 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:03.118681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18c135-9c16-44e7-b51e-2a4f7ae154df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:03.119307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18c135-9c16-44e7-b51e-2a4f7ae154df None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:03.119588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18c135-9c16-44e7-b51e-2a4f7ae154df None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:03.119863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18c135-9c16-44e7-b51e-2a4f7ae154df None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:03.120496 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18c135-9c16-44e7-b51e-2a4f7ae154df None admin] RBAC: Adding query filter params (name=image) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:03.121997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18c135-9c16-44e7-b51e-2a4f7ae154df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:03.122255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18c135-9c16-44e7-b51e-2a4f7ae154df None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:03.132136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 515/1006] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:03.133208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b5b1e-5452-486a-a369-0547e6bf0a7f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:03.148027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098828d-5a78-43dd-b435-f264d2c5d015 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:03.148325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098828d-5a78-43dd-b435-f264d2c5d015 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:03.148583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098828d-5a78-43dd-b435-f264d2c5d015 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:03.150151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 516/1007] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:03.156190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d17eb-c7e5-4ed7-9536-0770077ed328 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:03.156473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d17eb-c7e5-4ed7-9536-0770077ed328 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:03.156744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d17eb-c7e5-4ed7-9536-0770077ed328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:03.180957 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2e3d17eb-c7e5-4ed7-9536-0770077ed328 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:03.182321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2e3d17eb-c7e5-4ed7-9536-0770077ed328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:03.213582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:03.251410 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b5b1e-5452-486a-a369-0547e6bf0a7f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:03.264546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 517/1008] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:26:03.267453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:03.289299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e73c8-1b7a-486a-8933-ad764a9ca94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:03.290001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e73c8-1b7a-486a-8933-ad764a9ca94c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:03.290279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e73c8-1b7a-486a-8933-ad764a9ca94c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:03.290550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e73c8-1b7a-486a-8933-ad764a9ca94c None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:03.291222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e73c8-1b7a-486a-8933-ad764a9ca94c None admin] RBAC: Adding query filter params (type=image) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:03.291521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e73c8-1b7a-486a-8933-ad764a9ca94c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:03.292914 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e73c8-1b7a-486a-8933-ad764a9ca94c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:03.301296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 492/1009] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:03.539135 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d17eb-c7e5-4ed7-9536-0770077ed328 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:03.644892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d17eb-c7e5-4ed7-9536-0770077ed328 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:03.680849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e3f35-f0a2-4040-984e-69152cb6f8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:03.681514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e3f35-f0a2-4040-984e-69152cb6f8dd None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:03.681797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e3f35-f0a2-4040-984e-69152cb6f8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:03.682072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e3f35-f0a2-4040-984e-69152cb6f8dd None admin] PATH_INFO: `/v3/services/compute` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:03.682661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e3f35-f0a2-4040-984e-69152cb6f8dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:03.684019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e3f35-f0a2-4040-984e-69152cb6f8dd None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-e48e3f35-f0a2-4040-984e-69152cb6f8dd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:03.692809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:03.694084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 08:26:03.694930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:03.697174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 518/1010] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:26:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:03.699894 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e73c8-1b7a-486a-8933-ad764a9ca94c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:03.811528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e73c8-1b7a-486a-8933-ad764a9ca94c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:03.848313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae790f-71f9-47a7-a96a-e663e7ec7ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:03.849253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae790f-71f9-47a7-a96a-e663e7ec7ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:03.849719 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae790f-71f9-47a7-a96a-e663e7ec7ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:03.850173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae790f-71f9-47a7-a96a-e663e7ec7ed1 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:03.850998 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae790f-71f9-47a7-a96a-e663e7ec7ed1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:03.851482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae790f-71f9-47a7-a96a-e663e7ec7ed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:03.853015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae790f-71f9-47a7-a96a-e663e7ec7ed1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:03.863113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 493/1011] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:26:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:03.864576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e3f35-f0a2-4040-984e-69152cb6f8dd None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:03.892230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:03.971489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e3f35-f0a2-4040-984e-69152cb6f8dd None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:04.010851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c42c1d-9c13-4dde-ad75-e96aec624284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:04.011597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c42c1d-9c13-4dde-ad75-e96aec624284 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:04.011952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c42c1d-9c13-4dde-ad75-e96aec624284 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:04.012368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c42c1d-9c13-4dde-ad75-e96aec624284 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:04.013113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c42c1d-9c13-4dde-ad75-e96aec624284 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:04.013460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c42c1d-9c13-4dde-ad75-e96aec624284 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:04.015303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c42c1d-9c13-4dde-ad75-e96aec624284 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:04.025682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 519/1012] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 08:26:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:04.026662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae790f-71f9-47a7-a96a-e663e7ec7ed1 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:04.073587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:04.102736 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b5d34-e619-4e51-99c9-d79916a98b5d None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:04.103027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b5d34-e619-4e51-99c9-d79916a98b5d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:04.103303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b5d34-e619-4e51-99c9-d79916a98b5d None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:04.104649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 520/1013] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:26:04.110513 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d2f24-3d0e-4135-b827-777ac36873af None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:04.110798 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d2f24-3d0e-4135-b827-777ac36873af None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:04.111070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d2f24-3d0e-4135-b827-777ac36873af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:04.131761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2e3d2f24-3d0e-4135-b827-777ac36873af None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:04.133457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2e3d2f24-3d0e-4135-b827-777ac36873af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:04.135867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae790f-71f9-47a7-a96a-e663e7ec7ed1 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:04.176010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805798b1-f9e4-4a63-87f7-f09d55772048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:04.176747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805798b1-f9e4-4a63-87f7-f09d55772048 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:04.177031 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805798b1-f9e4-4a63-87f7-f09d55772048 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:04.177333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805798b1-f9e4-4a63-87f7-f09d55772048 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:04.178034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805798b1-f9e4-4a63-87f7-f09d55772048 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:04.180180 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805798b1-f9e4-4a63-87f7-f09d55772048 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:04.207110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 494/1014] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:26:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:26:04.221142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 521/1015] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:04 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:26:04.494300 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805798b1-f9e4-4a63-87f7-f09d55772048 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:04.609241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:04.614522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805798b1-f9e4-4a63-87f7-f09d55772048 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:04.652192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5982ae3-9416-41d2-95de-3797ef635178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:04.652837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5982ae3-9416-41d2-95de-3797ef635178 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:04.653141 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5982ae3-9416-41d2-95de-3797ef635178 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:04.653418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5982ae3-9416-41d2-95de-3797ef635178 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:04.654555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5982ae3-9416-41d2-95de-3797ef635178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:04.656557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5982ae3-9416-41d2-95de-3797ef635178 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:04.658478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 495/1016] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:04.664188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d2f24-3d0e-4135-b827-777ac36873af None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:04.680177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:04.769760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d2f24-3d0e-4135-b827-777ac36873af None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:04.813544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca846692-754d-4bf8-a329-c0c963507d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:04.814193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca846692-754d-4bf8-a329-c0c963507d0c None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:04.814549 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca846692-754d-4bf8-a329-c0c963507d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:04.814758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca846692-754d-4bf8-a329-c0c963507d0c None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:04.815448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca846692-754d-4bf8-a329-c0c963507d0c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:04.816911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca846692-754d-4bf8-a329-c0c963507d0c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:04.853127 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 522/1017] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:26:04 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:26:05.496605 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda84c0-c7e7-4254-a813-7c93feb3bbe7 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:05.497151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda84c0-c7e7-4254-a813-7c93feb3bbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:05.497265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda84c0-c7e7-4254-a813-7c93feb3bbe7 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:05.498556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 496/1018] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:05.504334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353577d-93f7-477e-8b27-ed79dfc412f8 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:05.504623 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353577d-93f7-477e-8b27-ed79dfc412f8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:05.504897 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353577d-93f7-477e-8b27-ed79dfc412f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:05.524377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b353577d-93f7-477e-8b27-ed79dfc412f8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:05.525619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b353577d-93f7-477e-8b27-ed79dfc412f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:05.591747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:05.605136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 523/1019] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:05 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:05.871949 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda84c0-c7e7-4254-a813-7c93feb3bbe7 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:05.975029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda84c0-c7e7-4254-a813-7c93feb3bbe7 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:06.008825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299e7a8-d5db-43f6-9943-befaa77ab86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:06.009466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299e7a8-d5db-43f6-9943-befaa77ab86c None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:06.009746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299e7a8-d5db-43f6-9943-befaa77ab86c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:06.010017 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299e7a8-d5db-43f6-9943-befaa77ab86c None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:06.010683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299e7a8-d5db-43f6-9943-befaa77ab86c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:06.012045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299e7a8-d5db-43f6-9943-befaa77ab86c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:06.029133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 497/1020] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:26:05 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:26:06.508955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2405d8-ba00-42a3-8e90-66d513697368 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:06.509533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2405d8-ba00-42a3-8e90-66d513697368 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:06.509533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2405d8-ba00-42a3-8e90-66d513697368 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:06.511031 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 524/1021] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:06.516370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d7daa-6495-49ac-aa33-dce36cd336a8 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:06.516806 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d7daa-6495-49ac-aa33-dce36cd336a8 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:06.516990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d7daa-6495-49ac-aa33-dce36cd336a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:06.536171 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-924d7daa-6495-49ac-aa33-dce36cd336a8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:06.537384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-924d7daa-6495-49ac-aa33-dce36cd336a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:06.558831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:06.616517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 498/1022] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:06.884493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2405d8-ba00-42a3-8e90-66d513697368 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:06.995233 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2405d8-ba00-42a3-8e90-66d513697368 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:07.030943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bbf6f-71c7-4409-944e-2cdd1f1d80c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:07.031583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bbf6f-71c7-4409-944e-2cdd1f1d80c9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:07.031861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bbf6f-71c7-4409-944e-2cdd1f1d80c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:07.032136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bbf6f-71c7-4409-944e-2cdd1f1d80c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:07.042371 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931bbf6f-71c7-4409-944e-2cdd1f1d80c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-931bbf6f-71c7-4409-944e-2cdd1f1d80c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:26:07.047684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:07.048686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:07.049566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:07.049566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:07.049566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:07.049566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:07.049566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:26:07.049566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:26:07.049566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:26:07.049566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:26:07.049566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:26:07.049566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:26:07.049566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 08:26:07.049566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:07.050977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 525/1023] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 08:26:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:07.055068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d7daa-6495-49ac-aa33-dce36cd336a8 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:07.091342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9d6f6-4e4f-42e1-8869-34208ab34c50 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:07.091652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9d6f6-4e4f-42e1-8869-34208ab34c50 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:07.091896 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9d6f6-4e4f-42e1-8869-34208ab34c50 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:07.093597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 526/1024] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:07.099991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48b942-661f-46b9-ac1a-f2d9cf754970 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:07.100260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48b942-661f-46b9-ac1a-f2d9cf754970 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:07.100695 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48b942-661f-46b9-ac1a-f2d9cf754970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:07.120176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2e48b942-661f-46b9-ac1a-f2d9cf754970 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:07.121395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2e48b942-661f-46b9-ac1a-f2d9cf754970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:07.157807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d7daa-6495-49ac-aa33-dce36cd336a8 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:07.175178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:07.193985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 527/1025] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:07.201760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0382af-392c-4d00-b333-8b6bc0e97d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:07.202421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0382af-392c-4d00-b333-8b6bc0e97d3e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:07.202701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0382af-392c-4d00-b333-8b6bc0e97d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:07.202973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0382af-392c-4d00-b333-8b6bc0e97d3e None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:07.203644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0382af-392c-4d00-b333-8b6bc0e97d3e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:07.203950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0382af-392c-4d00-b333-8b6bc0e97d3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:07.205340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0382af-392c-4d00-b333-8b6bc0e97d3e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:07.219053 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 499/1026] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 08:26:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:07.220079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48b942-661f-46b9-ac1a-f2d9cf754970 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:07.323589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48b942-661f-46b9-ac1a-f2d9cf754970 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:07.367332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4bed3-3002-4c8b-9aa8-5a4610ddb032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:07.367971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4bed3-3002-4c8b-9aa8-5a4610ddb032 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:07.368252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4bed3-3002-4c8b-9aa8-5a4610ddb032 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:07.368529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4bed3-3002-4c8b-9aa8-5a4610ddb032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:07.379526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4bed3-3002-4c8b-9aa8-5a4610ddb032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:07.380972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4bed3-3002-4c8b-9aa8-5a4610ddb032 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-c4d4bed3-3002-4c8b-9aa8-5a4610ddb032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:26:07.401830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:07.403165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:26:07.408983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:26:07.408983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:07.408983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:07.408983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 528/1027] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:07.409260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0382af-392c-4d00-b333-8b6bc0e97d3e None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:07.425640 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:07.476989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4bed3-3002-4c8b-9aa8-5a4610ddb032 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:07.536291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:07.537193 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0382af-392c-4d00-b333-8b6bc0e97d3e None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:07.576697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259acf6f-5cc5-4bac-9915-309f3d46b0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:07.577898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259acf6f-5cc5-4bac-9915-309f3d46b0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:07.578453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259acf6f-5cc5-4bac-9915-309f3d46b0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:07.578453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259acf6f-5cc5-4bac-9915-309f3d46b0c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:07.579061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259acf6f-5cc5-4bac-9915-309f3d46b0c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:07.579360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259acf6f-5cc5-4bac-9915-309f3d46b0c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:07.580711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259acf6f-5cc5-4bac-9915-309f3d46b0c9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:07.600710 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 500/1028] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:07.601931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259acf6f-5cc5-4bac-9915-309f3d46b0c9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:07.615313 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4bed3-3002-4c8b-9aa8-5a4610ddb032 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:07.654397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85044ad0-94c1-430f-88ff-49cb0dfd6557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:07.655366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85044ad0-94c1-430f-88ff-49cb0dfd6557 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:07.656489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85044ad0-94c1-430f-88ff-49cb0dfd6557 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:07.657007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85044ad0-94c1-430f-88ff-49cb0dfd6557 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:07.658348 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85044ad0-94c1-430f-88ff-49cb0dfd6557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:07.660026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85044ad0-94c1-430f-88ff-49cb0dfd6557 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:07.661800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 529/1029] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:07.667474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85044ad0-94c1-430f-88ff-49cb0dfd6557 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:07.685010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:07.712211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259acf6f-5cc5-4bac-9915-309f3d46b0c9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:07.749331 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997afc30-de93-4296-ae1d-3a00ecda4339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:07.749976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997afc30-de93-4296-ae1d-3a00ecda4339 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:07.750235 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997afc30-de93-4296-ae1d-3a00ecda4339 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:07.750507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997afc30-de93-4296-ae1d-3a00ecda4339 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:07.758503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997afc30-de93-4296-ae1d-3a00ecda4339 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:07.760750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997afc30-de93-4296-ae1d-3a00ecda4339 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-997afc30-de93-4296-ae1d-3a00ecda4339 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:26:07.774729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:07.775836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:07.776843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:07.776843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:07.776843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:26:07.776843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:26:07.776843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:26:07.776843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:26:07.776843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:26:07.776843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:26:07.776843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:26:07.776843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:07.780979 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 501/1030] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 08:26:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:26:07.782057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997afc30-de93-4296-ae1d-3a00ecda4339 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:07.795425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85044ad0-94c1-430f-88ff-49cb0dfd6557 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:07.803885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:07.840474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4ed5d-9e15-4f26-b8fb-8046a59b676a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:07.841491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4ed5d-9e15-4f26-b8fb-8046a59b676a None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:07.841922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4ed5d-9e15-4f26-b8fb-8046a59b676a None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:07.842398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4ed5d-9e15-4f26-b8fb-8046a59b676a None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:07.843409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4ed5d-9e15-4f26-b8fb-8046a59b676a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:07.846109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4ed5d-9e15-4f26-b8fb-8046a59b676a None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:07.872808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 530/1031] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:26:07 2022] POST /identity/v3/users => generated 285 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:26:07.907294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997afc30-de93-4296-ae1d-3a00ecda4339 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:07.947354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7b4ba-d8b2-4f6c-a8c3-09d30de98577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:07.947896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7b4ba-d8b2-4f6c-a8c3-09d30de98577 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:07.948178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7b4ba-d8b2-4f6c-a8c3-09d30de98577 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:07.948447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7b4ba-d8b2-4f6c-a8c3-09d30de98577 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:07.949219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7b4ba-d8b2-4f6c-a8c3-09d30de98577 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:07.949528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7b4ba-d8b2-4f6c-a8c3-09d30de98577 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:07.950972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7b4ba-d8b2-4f6c-a8c3-09d30de98577 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:07.953205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:07.973387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 502/1032] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 08:26:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:07.979936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4ed5d-9e15-4f26-b8fb-8046a59b676a None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:08.103036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4ed5d-9e15-4f26-b8fb-8046a59b676a None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:08.144443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95fddb-e527-415f-8a5e-bc443156c2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:08.145077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95fddb-e527-415f-8a5e-bc443156c2fe None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:08.145389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95fddb-e527-415f-8a5e-bc443156c2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:08.145667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95fddb-e527-415f-8a5e-bc443156c2fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:08.182834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:08.226753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95fddb-e527-415f-8a5e-bc443156c2fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:08.228242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95fddb-e527-415f-8a5e-bc443156c2fe None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:08.342224 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 531/1033] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:26:07 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:26:08.344571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7b4ba-d8b2-4f6c-a8c3-09d30de98577 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:08.360177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd0ef5-8180-41d6-b13d-cd1729927892 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:08.360466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd0ef5-8180-41d6-b13d-cd1729927892 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:08.360745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd0ef5-8180-41d6-b13d-cd1729927892 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:08.362268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 532/1034] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:26:08.367205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c885c-ca2c-439b-9a46-7215b397ec19 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:08.367499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c885c-ca2c-439b-9a46-7215b397ec19 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:08.367782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c885c-ca2c-439b-9a46-7215b397ec19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:08.388955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-fe3c885c-ca2c-439b-9a46-7215b397ec19 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:08.391050 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-fe3c885c-ca2c-439b-9a46-7215b397ec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:08.448578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7b4ba-d8b2-4f6c-a8c3-09d30de98577 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:08.455320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:08.469943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 533/1035] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:26:08.492981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c6da8-07a5-44ad-91ec-f171f9fb6802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:08.493874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c6da8-07a5-44ad-91ec-f171f9fb6802 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:08.494304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c6da8-07a5-44ad-91ec-f171f9fb6802 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:08.494724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c6da8-07a5-44ad-91ec-f171f9fb6802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:08.506001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c6da8-07a5-44ad-91ec-f171f9fb6802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:08.507601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c6da8-07a5-44ad-91ec-f171f9fb6802 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-c06c6da8-07a5-44ad-91ec-f171f9fb6802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:26:08.524543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:08.525727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:26:08.527391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:26:08.527391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:08.527391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:08.529222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 503/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:08.532763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c885c-ca2c-439b-9a46-7215b397ec19 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:08.636097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c885c-ca2c-439b-9a46-7215b397ec19 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:08.670480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b7eaf-65a0-44a8-8099-55b8aa5c0faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:08.671617 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b7eaf-65a0-44a8-8099-55b8aa5c0faa None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:08.671895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b7eaf-65a0-44a8-8099-55b8aa5c0faa None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:08.672167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b7eaf-65a0-44a8-8099-55b8aa5c0faa None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:08.672809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b7eaf-65a0-44a8-8099-55b8aa5c0faa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:08.673131 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b7eaf-65a0-44a8-8099-55b8aa5c0faa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:08.674490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b7eaf-65a0-44a8-8099-55b8aa5c0faa None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:08.686122 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 534/1037] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:08.692292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c6da8-07a5-44ad-91ec-f171f9fb6802 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:08.739772 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:08.740297 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b7eaf-65a0-44a8-8099-55b8aa5c0faa None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:08.797012 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:08.807163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c6da8-07a5-44ad-91ec-f171f9fb6802 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:08.843527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e821cd8-d9d3-4dce-ba19-109fb9843b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:08.844148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e821cd8-d9d3-4dce-ba19-109fb9843b89 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:08.844417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e821cd8-d9d3-4dce-ba19-109fb9843b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:08.844685 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e821cd8-d9d3-4dce-ba19-109fb9843b89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:08.849554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b7eaf-65a0-44a8-8099-55b8aa5c0faa None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:08.851456 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e821cd8-d9d3-4dce-ba19-109fb9843b89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:08.852942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e821cd8-d9d3-4dce-ba19-109fb9843b89 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-9e821cd8-d9d3-4dce-ba19-109fb9843b89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:26:08.865884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:08.867726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:08.868939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:26:08.868939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:26:08.868939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:26:08.868939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:26:08.868939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:26:08.868939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:26:08.868939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:26:08.868939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:08.868939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 504/1038] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:26:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:08.873400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e821cd8-d9d3-4dce-ba19-109fb9843b89 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:08.892349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b721f38-c24a-4350-a28a-fceed3791a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:08.892983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b721f38-c24a-4350-a28a-fceed3791a9c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:08.893297 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b721f38-c24a-4350-a28a-fceed3791a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:08.893631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b721f38-c24a-4350-a28a-fceed3791a9c None admin] PATH_INFO: `/v3/services/image` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:08.894212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b721f38-c24a-4350-a28a-fceed3791a9c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:08.895621 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b721f38-c24a-4350-a28a-fceed3791a9c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-3b721f38-c24a-4350-a28a-fceed3791a9c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:08.905197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:08.906272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:08.907119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 535/1039] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Sep 28 08:26:08 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:26:08.912168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b721f38-c24a-4350-a28a-fceed3791a9c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:08.980980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e821cd8-d9d3-4dce-ba19-109fb9843b89 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:08.983585 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:09.019829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51178fd-f42c-4292-a370-87ff910040f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:09.020437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51178fd-f42c-4292-a370-87ff910040f3 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:09.020714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51178fd-f42c-4292-a370-87ff910040f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:09.020981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51178fd-f42c-4292-a370-87ff910040f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:09.021631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b721f38-c24a-4350-a28a-fceed3791a9c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:09.021746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51178fd-f42c-4292-a370-87ff910040f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:09.021980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51178fd-f42c-4292-a370-87ff910040f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:09.023396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51178fd-f42c-4292-a370-87ff910040f3 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:09.036181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 505/1040] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:26:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:09.041230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:09.042786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51178fd-f42c-4292-a370-87ff910040f3 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:09.060113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be311912-b37a-4415-8691-ecf4041a50f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:09.060904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be311912-b37a-4415-8691-ecf4041a50f4 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:09.061211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be311912-b37a-4415-8691-ecf4041a50f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:09.061476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be311912-b37a-4415-8691-ecf4041a50f4 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:09.062712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be311912-b37a-4415-8691-ecf4041a50f4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:09.063009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be311912-b37a-4415-8691-ecf4041a50f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:09.064366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be311912-b37a-4415-8691-ecf4041a50f4 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:09.070148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 536/1041] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:09.075674 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be311912-b37a-4415-8691-ecf4041a50f4 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:09.158880 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51178fd-f42c-4292-a370-87ff910040f3 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:09.189204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be311912-b37a-4415-8691-ecf4041a50f4 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:09.199253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc97d7c-d6dc-4764-8903-5d3bc6f388f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:09.199860 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc97d7c-d6dc-4764-8903-5d3bc6f388f3 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:09.200129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc97d7c-d6dc-4764-8903-5d3bc6f388f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:09.200404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc97d7c-d6dc-4764-8903-5d3bc6f388f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:09.201357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc97d7c-d6dc-4764-8903-5d3bc6f388f3 None admin] RBAC: Adding query filter params (role.id=323909a5cc584ec1957e40e8928f214f, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=9c3c6efc3c154ff89066035cd9583459) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:09.201662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc97d7c-d6dc-4764-8903-5d3bc6f388f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:09.203248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc97d7c-d6dc-4764-8903-5d3bc6f388f3 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:09.209853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 506/1042] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:26:09 2022] GET /identity/v3/role_assignments?user.id=9c3c6efc3c154ff89066035cd9583459&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=323909a5cc584ec1957e40e8928f214f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:09.235824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcff81-65cd-4b66-9c3d-74a5818d3534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:09.236466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcff81-65cd-4b66-9c3d-74a5818d3534 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:09.236758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcff81-65cd-4b66-9c3d-74a5818d3534 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:09.237002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcff81-65cd-4b66-9c3d-74a5818d3534 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:09.237792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcff81-65cd-4b66-9c3d-74a5818d3534 None admin] RBAC: Adding query filter params (type=image) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:09.238101 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcff81-65cd-4b66-9c3d-74a5818d3534 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:09.239520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcff81-65cd-4b66-9c3d-74a5818d3534 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:09.246506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 537/1043] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:09.252778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc97d7c-d6dc-4764-8903-5d3bc6f388f3 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:09.285248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:09.366457 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc97d7c-d6dc-4764-8903-5d3bc6f388f3 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:09.403573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881d3c1-347c-4a6f-b21c-1be0fd1c7948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:09.404182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881d3c1-347c-4a6f-b21c-1be0fd1c7948 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:09.404464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881d3c1-347c-4a6f-b21c-1be0fd1c7948 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:09.404735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881d3c1-347c-4a6f-b21c-1be0fd1c7948 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:09.405541 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881d3c1-347c-4a6f-b21c-1be0fd1c7948 None admin] RBAC: Adding query filter params (interface=public, service_id=b228418b01a14f58bbb544bc6adc1713, region_id=RegionOne) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:09.405851 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881d3c1-347c-4a6f-b21c-1be0fd1c7948 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:09.407219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881d3c1-347c-4a6f-b21c-1be0fd1c7948 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:09.412921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 507/1044] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 08:26:09 2022] GET /identity/v3/endpoints?service_id=b228418b01a14f58bbb544bc6adc1713&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:09.418334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcff81-65cd-4b66-9c3d-74a5818d3534 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:09.466995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:09.532693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcff81-65cd-4b66-9c3d-74a5818d3534 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:09.580474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b4581-13e6-42d9-ae3b-7866b3585582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:09.581395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b4581-13e6-42d9-ae3b-7866b3585582 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:09.581824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b4581-13e6-42d9-ae3b-7866b3585582 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:09.582321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b4581-13e6-42d9-ae3b-7866b3585582 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:09.583081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b4581-13e6-42d9-ae3b-7866b3585582 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:09.584645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b4581-13e6-42d9-ae3b-7866b3585582 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:09.591741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 538/1045] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 08:26:09 2022] GET /identity/v3/services => generated 1957 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:26:10.176411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70aac81-1a95-4e19-a123-a746c590883e None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:10.176983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70aac81-1a95-4e19-a123-a746c590883e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:10.177212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70aac81-1a95-4e19-a123-a746c590883e None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:10.178515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 508/1046] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:10.183731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01b942-18e6-4854-a972-fc4bd55516a8 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:10.184156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01b942-18e6-4854-a972-fc4bd55516a8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:10.184316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01b942-18e6-4854-a972-fc4bd55516a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:10.204216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5f01b942-18e6-4854-a972-fc4bd55516a8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:10.205419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5f01b942-18e6-4854-a972-fc4bd55516a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:10.285980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 539/1047] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:10.554178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70aac81-1a95-4e19-a123-a746c590883e None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:10.649886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:10.662008 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70aac81-1a95-4e19-a123-a746c590883e None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:10.695581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc683bf-8ea5-4469-9d17-47ac154c8107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:10.696204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc683bf-8ea5-4469-9d17-47ac154c8107 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:10.696481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc683bf-8ea5-4469-9d17-47ac154c8107 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:10.696750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc683bf-8ea5-4469-9d17-47ac154c8107 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:10.707523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc683bf-8ea5-4469-9d17-47ac154c8107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-6fc683bf-8ea5-4469-9d17-47ac154c8107 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:26:10.712279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:10.713523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:10.714391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 509/1048] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:26:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:10.720008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01b942-18e6-4854-a972-fc4bd55516a8 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:10.760361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:10.829555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01b942-18e6-4854-a972-fc4bd55516a8 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:10.864475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee3996-e444-4f89-a097-e0639e3ecfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:10.865116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee3996-e444-4f89-a097-e0639e3ecfae None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:10.865399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee3996-e444-4f89-a097-e0639e3ecfae None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:10.865674 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee3996-e444-4f89-a097-e0639e3ecfae None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:10.866344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee3996-e444-4f89-a097-e0639e3ecfae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:10.866650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee3996-e444-4f89-a097-e0639e3ecfae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:10.868005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee3996-e444-4f89-a097-e0639e3ecfae None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:10.877458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 540/1049] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:26:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:10.883208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc683bf-8ea5-4469-9d17-47ac154c8107 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:10.985324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc683bf-8ea5-4469-9d17-47ac154c8107 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:11.019198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17336e3-586b-474c-8202-54221488e3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:11.019828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17336e3-586b-474c-8202-54221488e3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:11.020108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17336e3-586b-474c-8202-54221488e3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:11.020380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17336e3-586b-474c-8202-54221488e3d5 None admin] PATH_INFO: `/v3/users/f4fb9d7f53a048548e4b09618fb3887d` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:11.035121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17336e3-586b-474c-8202-54221488e3d5 None admin] RBAC: Authorizing `identity:get_user(user_id=f4fb9d7f53a048548e4b09618fb3887d)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:11.036557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17336e3-586b-474c-8202-54221488e3d5 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:11.038392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 510/1050] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:26:10 2022] GET /identity/v3/users/f4fb9d7f53a048548e4b09618fb3887d => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:11.044083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee3996-e444-4f89-a097-e0639e3ecfae None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:11.147183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:11.150789 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee3996-e444-4f89-a097-e0639e3ecfae None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:11.190474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc19f63-7fd1-4668-9aa1-c7673fd81899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:11.191102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19f63-7fd1-4668-9aa1-c7673fd81899 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:11.191355 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19f63-7fd1-4668-9aa1-c7673fd81899 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:11.191632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19f63-7fd1-4668-9aa1-c7673fd81899 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:11.260412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc19f63-7fd1-4668-9aa1-c7673fd81899 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:11.261940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc19f63-7fd1-4668-9aa1-c7673fd81899 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:11.364480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:11.373929 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 541/1051] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:26:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 08:26:11.376134 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17336e3-586b-474c-8202-54221488e3d5 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:11.390272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:11.425244 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d64786-3e39-4b4c-8ded-0764455a6140 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:11.425443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d64786-3e39-4b4c-8ded-0764455a6140 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:11.425717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d64786-3e39-4b4c-8ded-0764455a6140 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:11.427065 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 542/1052] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:26:11.432090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b801d3c-af7a-4c32-a980-8c569c4e02cd None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:11.432374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b801d3c-af7a-4c32-a980-8c569c4e02cd None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:11.432650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b801d3c-af7a-4c32-a980-8c569c4e02cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:11.450752 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-1b801d3c-af7a-4c32-a980-8c569c4e02cd None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:11.451935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1b801d3c-af7a-4c32-a980-8c569c4e02cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:11.484632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17336e3-586b-474c-8202-54221488e3d5 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:11.520495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ddf32-3688-49a9-9dc5-38eb76d17997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:11.521563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ddf32-3688-49a9-9dc5-38eb76d17997 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:11.521906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ddf32-3688-49a9-9dc5-38eb76d17997 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:11.522246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ddf32-3688-49a9-9dc5-38eb76d17997 None admin] PATH_INFO: `/v3/projects/e37f1342ee5f476e8e769844e4f228a6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:11.524270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ddf32-3688-49a9-9dc5-38eb76d17997 None admin] RBAC: Authorizing `identity:get_project(project_id=e37f1342ee5f476e8e769844e4f228a6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:11.526102 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ddf32-3688-49a9-9dc5-38eb76d17997 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:11.533361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 511/1053] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:26:11 2022] GET /identity/v3/projects/e37f1342ee5f476e8e769844e4f228a6 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:11.533503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 543/1054] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:26:11.537869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ddf32-3688-49a9-9dc5-38eb76d17997 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:11.620125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:11.643752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ddf32-3688-49a9-9dc5-38eb76d17997 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:11.680388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7cc79-6ce5-402b-bc20-4e79369b03d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:11.680926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7cc79-6ce5-402b-bc20-4e79369b03d2 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:11.681262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7cc79-6ce5-402b-bc20-4e79369b03d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:11.681528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7cc79-6ce5-402b-bc20-4e79369b03d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:11.682369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7cc79-6ce5-402b-bc20-4e79369b03d2 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=e37f1342ee5f476e8e769844e4f228a6, user.id=f4fb9d7f53a048548e4b09618fb3887d) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:11.682570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7cc79-6ce5-402b-bc20-4e79369b03d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:11.684026 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb7cc79-6ce5-402b-bc20-4e79369b03d2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:11.691331 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 512/1055] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:26:11 2022] GET /identity/v3/role_assignments?user.id=f4fb9d7f53a048548e4b09618fb3887d&scope.project.id=e37f1342ee5f476e8e769844e4f228a6&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:11.799345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b801d3c-af7a-4c32-a980-8c569c4e02cd None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:11.803575 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aacfe3-dd9d-4af1-afa4-7ca7aaf9865c None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:11.803742 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aacfe3-dd9d-4af1-afa4-7ca7aaf9865c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:11.804014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aacfe3-dd9d-4af1-afa4-7ca7aaf9865c None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:11.805569 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 513/1056] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:11.811025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b16c7-68d2-4e26-a10d-772bce542a56 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:11.811345 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b16c7-68d2-4e26-a10d-772bce542a56 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:11.812113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b16c7-68d2-4e26-a10d-772bce542a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:11.831917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-af9b16c7-68d2-4e26-a10d-772bce542a56 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:11.833195 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-af9b16c7-68d2-4e26-a10d-772bce542a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:11.909139 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 514/1057] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:26:11.926411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b801d3c-af7a-4c32-a980-8c569c4e02cd None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:11.959120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:11.968192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa8e9e-1892-4264-bc33-439a790d4b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:11.968827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa8e9e-1892-4264-bc33-439a790d4b50 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:11.969574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa8e9e-1892-4264-bc33-439a790d4b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:11.969574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa8e9e-1892-4264-bc33-439a790d4b50 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:11.980618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa8e9e-1892-4264-bc33-439a790d4b50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-92aa8e9e-1892-4264-bc33-439a790d4b50 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:26:11.985526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:11.986448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:11.987732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 544/1058] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 08:26:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:11.992560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b16c7-68d2-4e26-a10d-772bce542a56 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:12.078464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:12.110068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b16c7-68d2-4e26-a10d-772bce542a56 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:12.145382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5afcd9-dcbd-4808-9380-ce513dfd3a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:12.145994 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5afcd9-dcbd-4808-9380-ce513dfd3a81 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:12.146268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5afcd9-dcbd-4808-9380-ce513dfd3a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:12.146537 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5afcd9-dcbd-4808-9380-ce513dfd3a81 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:12.147266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5afcd9-dcbd-4808-9380-ce513dfd3a81 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:12.147567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5afcd9-dcbd-4808-9380-ce513dfd3a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:12.148972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5afcd9-dcbd-4808-9380-ce513dfd3a81 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:12.164136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 515/1059] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 08:26:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:12.165669 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa8e9e-1892-4264-bc33-439a790d4b50 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:12.206517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5afcd9-dcbd-4808-9380-ce513dfd3a81 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:12.276856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa8e9e-1892-4264-bc33-439a790d4b50 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:12.316831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706a044-537c-4f4b-8542-ac79d9e00e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:12.317944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706a044-537c-4f4b-8542-ac79d9e00e81 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:12.318429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706a044-537c-4f4b-8542-ac79d9e00e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:12.318911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706a044-537c-4f4b-8542-ac79d9e00e81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:12.330911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5afcd9-dcbd-4808-9380-ce513dfd3a81 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:12.334195 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706a044-537c-4f4b-8542-ac79d9e00e81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:12.336158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706a044-537c-4f4b-8542-ac79d9e00e81 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-a706a044-537c-4f4b-8542-ac79d9e00e81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:26:12.352272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:12.353300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:26:12.354193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:26:12.354193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:12.354193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:12.354193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 545/1060] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:12.362531 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706a044-537c-4f4b-8542-ac79d9e00e81 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:12.370185 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd41847-3ad8-4d4f-865d-f3d2be1eef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:12.370820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd41847-3ad8-4d4f-865d-f3d2be1eef23 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:12.371099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd41847-3ad8-4d4f-865d-f3d2be1eef23 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:12.371369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd41847-3ad8-4d4f-865d-f3d2be1eef23 None admin] PATH_INFO: `/v3/services/image` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:12.371949 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd41847-3ad8-4d4f-865d-f3d2be1eef23 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:12.373600 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd41847-3ad8-4d4f-865d-f3d2be1eef23 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:12.375398 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-8cd41847-3ad8-4d4f-865d-f3d2be1eef23 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:12.383840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:12.385526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 08:26:12.386520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:12.394659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 516/1061] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Sep 28 08:26:12 2022] GET /identity/v3/services/image => generated 86 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:26:12.395509 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd41847-3ad8-4d4f-865d-f3d2be1eef23 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:12.415515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:12.479849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706a044-537c-4f4b-8542-ac79d9e00e81 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:12.509151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd41847-3ad8-4d4f-865d-f3d2be1eef23 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:12.522399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73440d-484c-431a-ae89-c77aa96ffa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:12.523099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73440d-484c-431a-ae89-c77aa96ffa18 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:12.523447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73440d-484c-431a-ae89-c77aa96ffa18 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:12.523801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73440d-484c-431a-ae89-c77aa96ffa18 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:12.524520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73440d-484c-431a-ae89-c77aa96ffa18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:12.524897 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73440d-484c-431a-ae89-c77aa96ffa18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:12.526407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73440d-484c-431a-ae89-c77aa96ffa18 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:12.541016 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 546/1062] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:12.546810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73440d-484c-431a-ae89-c77aa96ffa18 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:12.554702 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03799cca-bca9-4de8-9445-18499b64a79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:12.555366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03799cca-bca9-4de8-9445-18499b64a79c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:12.555648 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03799cca-bca9-4de8-9445-18499b64a79c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:12.555922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03799cca-bca9-4de8-9445-18499b64a79c None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:12.556554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03799cca-bca9-4de8-9445-18499b64a79c None admin] RBAC: Adding query filter params (name=image) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:12.556857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03799cca-bca9-4de8-9445-18499b64a79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:12.558313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03799cca-bca9-4de8-9445-18499b64a79c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:12.563640 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 517/1063] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:12.569603 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03799cca-bca9-4de8-9445-18499b64a79c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:12.642014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:12.661312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73440d-484c-431a-ae89-c77aa96ffa18 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:12.662975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:12.682266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03799cca-bca9-4de8-9445-18499b64a79c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:12.700746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c561f5-c7a2-432a-94e1-f95323e8d36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:12.701403 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c561f5-c7a2-432a-94e1-f95323e8d36c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:12.701682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c561f5-c7a2-432a-94e1-f95323e8d36c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:12.701952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c561f5-c7a2-432a-94e1-f95323e8d36c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:12.710005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c561f5-c7a2-432a-94e1-f95323e8d36c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:12.711547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c561f5-c7a2-432a-94e1-f95323e8d36c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:12.719381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d4c2b-e371-4ace-89f0-f404215a3a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:12.719985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d4c2b-e371-4ace-89f0-f404215a3a67 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:12.720324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d4c2b-e371-4ace-89f0-f404215a3a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:12.720602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d4c2b-e371-4ace-89f0-f404215a3a67 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:12.721263 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d4c2b-e371-4ace-89f0-f404215a3a67 None admin] RBAC: Adding query filter params (type=image) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:12.721560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d4c2b-e371-4ace-89f0-f404215a3a67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:12.722907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d4c2b-e371-4ace-89f0-f404215a3a67 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-c8c561f5-c7a2-432a-94e1-f95323e8d36c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:26:12.724838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:12.727111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:12.727111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:12.727111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:26:12.727111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:26:12.727111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:26:12.727111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:26:12.727111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:26:12.727111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:26:12.727111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:26:12.727111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:12.728122 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 547/1064] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 08:26:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:26:12.736412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 518/1065] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:12.736590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c561f5-c7a2-432a-94e1-f95323e8d36c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:12.739029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d4c2b-e371-4ace-89f0-f404215a3a67 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:12.843602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c561f5-c7a2-432a-94e1-f95323e8d36c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:12.859305 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d4c2b-e371-4ace-89f0-f404215a3a67 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:12.888883 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0fafb-6953-4b4e-b540-4a2daeb2efcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:12.889720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0fafb-6953-4b4e-b540-4a2daeb2efcf None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:12.890138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0fafb-6953-4b4e-b540-4a2daeb2efcf None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:12.890762 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0fafb-6953-4b4e-b540-4a2daeb2efcf None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:12.891638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0fafb-6953-4b4e-b540-4a2daeb2efcf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:12.892496 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0fafb-6953-4b4e-b540-4a2daeb2efcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:12.894340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0fafb-6953-4b4e-b540-4a2daeb2efcf None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:12.895008 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:12.908724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c371dcd-0ee3-45ce-af1d-f7d0b939fad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:12.909634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c371dcd-0ee3-45ce-af1d-f7d0b939fad0 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:12.910077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c371dcd-0ee3-45ce-af1d-f7d0b939fad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:12.910508 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c371dcd-0ee3-45ce-af1d-f7d0b939fad0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:12.911343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c371dcd-0ee3-45ce-af1d-f7d0b939fad0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:12.913557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c371dcd-0ee3-45ce-af1d-f7d0b939fad0 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:12.922365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 548/1066] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 08:26:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:12.923403 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0fafb-6953-4b4e-b540-4a2daeb2efcf None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:12.940780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 519/1067] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:26:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:26:12.957170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:13.070114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0fafb-6953-4b4e-b540-4a2daeb2efcf None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:13.118824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00d63d-985b-4e7f-bd97-ba1cc91edff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:13.119383 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00d63d-985b-4e7f-bd97-ba1cc91edff1 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:13.119665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00d63d-985b-4e7f-bd97-ba1cc91edff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:13.119938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00d63d-985b-4e7f-bd97-ba1cc91edff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:13.131458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00d63d-985b-4e7f-bd97-ba1cc91edff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:13.132963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00d63d-985b-4e7f-bd97-ba1cc91edff1 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-0f00d63d-985b-4e7f-bd97-ba1cc91edff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:26:13.149672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:13.150917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:26:13.152362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:26:13.152362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:13.152362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:13.153626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 549/1068] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:13.159447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c371dcd-0ee3-45ce-af1d-f7d0b939fad0 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:13.286574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c371dcd-0ee3-45ce-af1d-f7d0b939fad0 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:13.321851 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b808e-4c9b-4df0-91d3-0f217fea0cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:13.322464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b808e-4c9b-4df0-91d3-0f217fea0cfd None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:13.322744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b808e-4c9b-4df0-91d3-0f217fea0cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:13.323016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b808e-4c9b-4df0-91d3-0f217fea0cfd None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:13.323652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b808e-4c9b-4df0-91d3-0f217fea0cfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:13.323956 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b808e-4c9b-4df0-91d3-0f217fea0cfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:13.325364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b808e-4c9b-4df0-91d3-0f217fea0cfd None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:13.337686 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 520/1069] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:13.343650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00d63d-985b-4e7f-bd97-ba1cc91edff1 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:13.413057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:13.449409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00d63d-985b-4e7f-bd97-ba1cc91edff1 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:13.484385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bb6ff-94d3-4ec3-8465-631d1158c03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:13.485024 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bb6ff-94d3-4ec3-8465-631d1158c03d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:13.485437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bb6ff-94d3-4ec3-8465-631d1158c03d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:13.485618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bb6ff-94d3-4ec3-8465-631d1158c03d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:13.492590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bb6ff-94d3-4ec3-8465-631d1158c03d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:13.494063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bb6ff-94d3-4ec3-8465-631d1158c03d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-379bb6ff-94d3-4ec3-8465-631d1158c03d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:26:13.507003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:13.507994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:13.508898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:26:13.508898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:26:13.508898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:26:13.508898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:26:13.508898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:26:13.508898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:26:13.508898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:26:13.508898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:13.508898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 550/1070] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:26:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:13.513922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b808e-4c9b-4df0-91d3-0f217fea0cfd None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:13.562661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:13.627365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b808e-4c9b-4df0-91d3-0f217fea0cfd None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:13.671995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217320f8-1878-408f-a2bc-2b921e3ad53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:13.672522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217320f8-1878-408f-a2bc-2b921e3ad53d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:13.672798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217320f8-1878-408f-a2bc-2b921e3ad53d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:13.673069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217320f8-1878-408f-a2bc-2b921e3ad53d None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:13.673797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217320f8-1878-408f-a2bc-2b921e3ad53d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:13.674102 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217320f8-1878-408f-a2bc-2b921e3ad53d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:13.675534 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217320f8-1878-408f-a2bc-2b921e3ad53d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:13.692288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 521/1071] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:26:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:13.693510 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bb6ff-94d3-4ec3-8465-631d1158c03d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:13.802796 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bb6ff-94d3-4ec3-8465-631d1158c03d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:13.830230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:13.839554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893051c-70c6-4459-b2ed-4eadcb1328f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:13.840225 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893051c-70c6-4459-b2ed-4eadcb1328f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:13.840509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893051c-70c6-4459-b2ed-4eadcb1328f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:13.840777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893051c-70c6-4459-b2ed-4eadcb1328f3 None admin] PATH_INFO: `/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/9c3c6efc3c154ff89066035cd9583459/roles/323909a5cc584ec1957e40e8928f214f` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:13.843643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893051c-70c6-4459-b2ed-4eadcb1328f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca4d4801f2f40419f6881984fbbfbcd, user_id=9c3c6efc3c154ff89066035cd9583459, role_id=323909a5cc584ec1957e40e8928f214f)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:13.845057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893051c-70c6-4459-b2ed-4eadcb1328f3 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:13.865117 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 551/1072] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:26:13 2022] PUT /identity/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/9c3c6efc3c154ff89066035cd9583459/roles/323909a5cc584ec1957e40e8928f214f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:26:13.877491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290976b-ec5a-4805-9b17-735f8f69830e None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:13.877826 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290976b-ec5a-4805-9b17-735f8f69830e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:13.878108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290976b-ec5a-4805-9b17-735f8f69830e None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:13.879508 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 522/1073] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:13.885501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1eb2c1-9b9b-43e0-9755-bda8db3e0a2c None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:13.885789 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1eb2c1-9b9b-43e0-9755-bda8db3e0a2c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:13.886065 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1eb2c1-9b9b-43e0-9755-bda8db3e0a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:13.907036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-1d1eb2c1-9b9b-43e0-9755-bda8db3e0a2c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:13.908342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1d1eb2c1-9b9b-43e0-9755-bda8db3e0a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:13.987375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 552/1074] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:14.256325 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290976b-ec5a-4805-9b17-735f8f69830e None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:14.363873 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290976b-ec5a-4805-9b17-735f8f69830e None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:14.365453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:14.401456 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9b33a-5219-4994-b26c-bebf8f92d09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:14.402081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9b33a-5219-4994-b26c-bebf8f92d09b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:14.402598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9b33a-5219-4994-b26c-bebf8f92d09b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:14.402862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9b33a-5219-4994-b26c-bebf8f92d09b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:14.414331 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9b33a-5219-4994-b26c-bebf8f92d09b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-bae9b33a-5219-4994-b26c-bebf8f92d09b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:26:14.419163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:14.421276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:14.422191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 523/1075] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:26:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:14.426319 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1eb2c1-9b9b-43e0-9755-bda8db3e0a2c None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:14.537563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1eb2c1-9b9b-43e0-9755-bda8db3e0a2c None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:14.555228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:14.578499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d65149-9468-413f-a691-482967c89386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:14.579117 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d65149-9468-413f-a691-482967c89386 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:14.579398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d65149-9468-413f-a691-482967c89386 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:14.579674 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d65149-9468-413f-a691-482967c89386 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:14.580353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d65149-9468-413f-a691-482967c89386 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:14.580660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d65149-9468-413f-a691-482967c89386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:14.582052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d65149-9468-413f-a691-482967c89386 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:14.597139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 553/1076] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:26:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:14.599010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9b33a-5219-4994-b26c-bebf8f92d09b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:14.703945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9b33a-5219-4994-b26c-bebf8f92d09b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:14.739084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f179f7-2cd9-438c-a585-1ee60273639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:14.739836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f179f7-2cd9-438c-a585-1ee60273639d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:14.740219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f179f7-2cd9-438c-a585-1ee60273639d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:14.740728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f179f7-2cd9-438c-a585-1ee60273639d None admin] PATH_INFO: `/v3/users/f4fb9d7f53a048548e4b09618fb3887d` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:14.741966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f179f7-2cd9-438c-a585-1ee60273639d None admin] RBAC: Authorizing `identity:get_user(user_id=f4fb9d7f53a048548e4b09618fb3887d)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:14.743488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f179f7-2cd9-438c-a585-1ee60273639d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:14.749758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 524/1077] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:26:14 2022] GET /identity/v3/users/f4fb9d7f53a048548e4b09618fb3887d => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:14.751293 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d65149-9468-413f-a691-482967c89386 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:14.861610 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d65149-9468-413f-a691-482967c89386 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:14.894569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b77c2-bcd1-4328-ad2c-8f776e4679ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:14.895192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b77c2-bcd1-4328-ad2c-8f776e4679ba None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:14.895468 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b77c2-bcd1-4328-ad2c-8f776e4679ba None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:14.895739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b77c2-bcd1-4328-ad2c-8f776e4679ba None admin] PATH_INFO: `/v3/projects/e37f1342ee5f476e8e769844e4f228a6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:14.896834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b77c2-bcd1-4328-ad2c-8f776e4679ba None admin] RBAC: Authorizing `identity:get_project(project_id=e37f1342ee5f476e8e769844e4f228a6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:14.898399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b77c2-bcd1-4328-ad2c-8f776e4679ba None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:14.905170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 554/1078] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:26:14 2022] GET /identity/v3/projects/e37f1342ee5f476e8e769844e4f228a6 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:14.907096 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f179f7-2cd9-438c-a585-1ee60273639d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:14.940223 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:15.012947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f179f7-2cd9-438c-a585-1ee60273639d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:15.056796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb12bd-31e4-4c39-a793-778507efa5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:15.057667 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb12bd-31e4-4c39-a793-778507efa5c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:15.058423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb12bd-31e4-4c39-a793-778507efa5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:15.059229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb12bd-31e4-4c39-a793-778507efa5c7 None admin] PATH_INFO: `/v3/projects/e37f1342ee5f476e8e769844e4f228a6/users/f4fb9d7f53a048548e4b09618fb3887d/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:15.062601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb12bd-31e4-4c39-a793-778507efa5c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e37f1342ee5f476e8e769844e4f228a6, user_id=f4fb9d7f53a048548e4b09618fb3887d, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:15.064152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb12bd-31e4-4c39-a793-778507efa5c7 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:15.079275 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 525/1079] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:26:14 2022] PUT /identity/v3/projects/e37f1342ee5f476e8e769844e4f228a6/users/f4fb9d7f53a048548e4b09618fb3887d/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:26:15.232968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb884007-c071-4457-b283-dfce76c6a7d3 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:15.233344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb884007-c071-4457-b283-dfce76c6a7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:15.233607 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb884007-c071-4457-b283-dfce76c6a7d3 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:15.235125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 555/1080] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:15.240659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac0ce4-83e0-4772-afa7-06310650aa0c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:15.241084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac0ce4-83e0-4772-afa7-06310650aa0c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:15.241553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac0ce4-83e0-4772-afa7-06310650aa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:15.261278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-64ac0ce4-83e0-4772-afa7-06310650aa0c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:15.262573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-64ac0ce4-83e0-4772-afa7-06310650aa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:15.333616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 526/1081] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:15.606829 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb884007-c071-4457-b283-dfce76c6a7d3 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:15.660232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:15.713544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb884007-c071-4457-b283-dfce76c6a7d3 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:15.748541 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a7ed0-56ea-458a-9252-6f1e0b23d6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:15.749221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a7ed0-56ea-458a-9252-6f1e0b23d6db None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:15.749500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a7ed0-56ea-458a-9252-6f1e0b23d6db None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:15.749771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a7ed0-56ea-458a-9252-6f1e0b23d6db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:15.818942 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a7ed0-56ea-458a-9252-6f1e0b23d6db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:15.820290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a7ed0-56ea-458a-9252-6f1e0b23d6db None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:15.868033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:15.921295 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 556/1082] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:26:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:26:15.927042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac0ce4-83e0-4772-afa7-06310650aa0c None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:15.949565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:16.038234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac0ce4-83e0-4772-afa7-06310650aa0c None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:16.074208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ab0cb-a8d9-44c2-8023-6dcb1164d09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:16.074744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ab0cb-a8d9-44c2-8023-6dcb1164d09d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:16.075023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ab0cb-a8d9-44c2-8023-6dcb1164d09d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:16.075294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ab0cb-a8d9-44c2-8023-6dcb1164d09d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:16.084172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ac1e-7274-45fd-9915-2f26e4be9209 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:16.084460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ac1e-7274-45fd-9915-2f26e4be9209 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:16.084738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ac1e-7274-45fd-9915-2f26e4be9209 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:16.085711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ab0cb-a8d9-44c2-8023-6dcb1164d09d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:16.086287 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 557/1083] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:26:16.087208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ab0cb-a8d9-44c2-8023-6dcb1164d09d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:16.091102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073f944-539d-4ade-8757-1029658a7331 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:16.091389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073f944-539d-4ade-8757-1029658a7331 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:16.091664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073f944-539d-4ade-8757-1029658a7331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-e78ab0cb-a8d9-44c2-8023-6dcb1164d09d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:26:16.103986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:16.104940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:26:16.112078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:26:16.112078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:16.112078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:16.112078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 527/1084] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:16.112363 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5073f944-539d-4ade-8757-1029658a7331 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:16.112363 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5073f944-539d-4ade-8757-1029658a7331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:16.115451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ab0cb-a8d9-44c2-8023-6dcb1164d09d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:16.184643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 558/1085] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:26:16.198840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:16.232258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ab0cb-a8d9-44c2-8023-6dcb1164d09d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:16.267246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20dde06-3d78-4d41-a28d-a5072c51f056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:16.267860 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dde06-3d78-4d41-a28d-a5072c51f056 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:16.268140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dde06-3d78-4d41-a28d-a5072c51f056 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:16.268413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dde06-3d78-4d41-a28d-a5072c51f056 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:16.269047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20dde06-3d78-4d41-a28d-a5072c51f056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:16.269375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20dde06-3d78-4d41-a28d-a5072c51f056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:16.270725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20dde06-3d78-4d41-a28d-a5072c51f056 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:16.289140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 528/1086] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:16.457612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073f944-539d-4ade-8757-1029658a7331 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:16.551064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:16.576483 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073f944-539d-4ade-8757-1029658a7331 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:16.612910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a61a23-7aaa-4c82-a462-5372ac2f69cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:16.613591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a61a23-7aaa-4c82-a462-5372ac2f69cd None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:16.613858 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a61a23-7aaa-4c82-a462-5372ac2f69cd None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:16.614096 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a61a23-7aaa-4c82-a462-5372ac2f69cd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:16.625195 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a61a23-7aaa-4c82-a462-5372ac2f69cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-c5a61a23-7aaa-4c82-a462-5372ac2f69cd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:26:16.629955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:16.631083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:16.631956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 559/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 08:26:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:26:16.637376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20dde06-3d78-4d41-a28d-a5072c51f056 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:16.737159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20dde06-3d78-4d41-a28d-a5072c51f056 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:16.767074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:16.772200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbaf18-b509-4261-9dd2-e5d841271238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:16.772812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbaf18-b509-4261-9dd2-e5d841271238 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:16.773103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbaf18-b509-4261-9dd2-e5d841271238 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:16.773388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbaf18-b509-4261-9dd2-e5d841271238 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:16.774111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbaf18-b509-4261-9dd2-e5d841271238 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:16.774480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbaf18-b509-4261-9dd2-e5d841271238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:16.775842 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbaf18-b509-4261-9dd2-e5d841271238 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:16.784375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 529/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 08:26:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:16.790203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a61a23-7aaa-4c82-a462-5372ac2f69cd None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:16.892891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a61a23-7aaa-4c82-a462-5372ac2f69cd None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:16.927840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96b456-0680-4de5-b418-eb7c45d19d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:16.928467 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96b456-0680-4de5-b418-eb7c45d19d25 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:16.928746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96b456-0680-4de5-b418-eb7c45d19d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:16.929018 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96b456-0680-4de5-b418-eb7c45d19d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:16.935982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:16.941123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96b456-0680-4de5-b418-eb7c45d19d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:16.942600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96b456-0680-4de5-b418-eb7c45d19d25 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-3d96b456-0680-4de5-b418-eb7c45d19d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:26:16.957243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:16.958250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:26:16.959704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:26:16.959704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:16.959704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:16.959704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 560/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:26:16.964126 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbaf18-b509-4261-9dd2-e5d841271238 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:17.064151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbaf18-b509-4261-9dd2-e5d841271238 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:17.097111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd0b9d-3f37-4edb-b668-bb155967b6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:17.097729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd0b9d-3f37-4edb-b668-bb155967b6ab None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:17.098004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd0b9d-3f37-4edb-b668-bb155967b6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:17.098276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd0b9d-3f37-4edb-b668-bb155967b6ab None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:17.098900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd0b9d-3f37-4edb-b668-bb155967b6ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:17.099197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd0b9d-3f37-4edb-b668-bb155967b6ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:17.100544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd0b9d-3f37-4edb-b668-bb155967b6ab None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:17.113505 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 530/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:17.118963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96b456-0680-4de5-b418-eb7c45d19d25 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:17.225771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96b456-0680-4de5-b418-eb7c45d19d25 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:17.259303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c237cf-ae01-4086-a787-a09d86cb1975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:17.259956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c237cf-ae01-4086-a787-a09d86cb1975 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:17.260232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c237cf-ae01-4086-a787-a09d86cb1975 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:17.260504 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c237cf-ae01-4086-a787-a09d86cb1975 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:17.268041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c237cf-ae01-4086-a787-a09d86cb1975 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:17.269490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c237cf-ae01-4086-a787-a09d86cb1975 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-93c237cf-ae01-4086-a787-a09d86cb1975 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:26:17.281099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:17.282157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:17.283118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:17.283118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:17.283118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:26:17.283118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:26:17.283118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:26:17.283118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:26:17.283118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:26:17.283118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:26:17.283118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 08:26:17.283118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:17.283118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 561/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 08:26:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:17.287977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c237cf-ae01-4086-a787-a09d86cb1975 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:17.288647 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d5575-8a24-49c9-9a43-e89a5cd978b4 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:17.288939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d5575-8a24-49c9-9a43-e89a5cd978b4 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:17.289260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d5575-8a24-49c9-9a43-e89a5cd978b4 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:17.290597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 531/1092] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:17.295707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fca4c-6f22-49bd-8d3f-b924cb6240e3 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:17.296532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fca4c-6f22-49bd-8d3f-b924cb6240e3 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:17.296807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fca4c-6f22-49bd-8d3f-b924cb6240e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:17.314615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ec5fca4c-6f22-49bd-8d3f-b924cb6240e3 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:17.315724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ec5fca4c-6f22-49bd-8d3f-b924cb6240e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:17.338161 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:17.365388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:17.383595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 532/1093] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:26:17.390050 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c237cf-ae01-4086-a787-a09d86cb1975 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:17.423741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1082c-256b-49f4-abbc-750e1591090f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:17.424348 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1082c-256b-49f4-abbc-750e1591090f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:17.424627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1082c-256b-49f4-abbc-750e1591090f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:17.424897 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1082c-256b-49f4-abbc-750e1591090f None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:17.425653 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1082c-256b-49f4-abbc-750e1591090f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:17.425962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1082c-256b-49f4-abbc-750e1591090f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:17.427374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1082c-256b-49f4-abbc-750e1591090f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:17.444615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 562/1094] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 08:26:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:17.450434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fca4c-6f22-49bd-8d3f-b924cb6240e3 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:17.552282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fca4c-6f22-49bd-8d3f-b924cb6240e3 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:17.586703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6d240-f145-47ec-a628-b1eac138b07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:17.587381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6d240-f145-47ec-a628-b1eac138b07c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:17.587503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6d240-f145-47ec-a628-b1eac138b07c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:17.587767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6d240-f145-47ec-a628-b1eac138b07c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:17.640323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:17.650457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1082c-256b-49f4-abbc-750e1591090f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:17.658600 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6d240-f145-47ec-a628-b1eac138b07c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:17.660057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6d240-f145-47ec-a628-b1eac138b07c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:17.751782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1082c-256b-49f4-abbc-750e1591090f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:17.759139 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 533/1095] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:26:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 08:26:17.761382 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:17.765693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6d240-f145-47ec-a628-b1eac138b07c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:17.787407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3da36-229e-4a94-a513-de48b59a268a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:17.788042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3da36-229e-4a94-a513-de48b59a268a None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:17.788325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3da36-229e-4a94-a513-de48b59a268a None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:17.788598 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3da36-229e-4a94-a513-de48b59a268a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:17.798857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3da36-229e-4a94-a513-de48b59a268a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-5cd3da36-229e-4a94-a513-de48b59a268a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:26:17.803696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:17.804771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:17.805734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 563/1096] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:26:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:26:17.810731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3da36-229e-4a94-a513-de48b59a268a None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:17.852586 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:17.869576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6d240-f145-47ec-a628-b1eac138b07c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:17.905095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5793e-ccec-4cbb-a4bf-dc77dd9038a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:17.905741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5793e-ccec-4cbb-a4bf-dc77dd9038a1 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:17.906017 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5793e-ccec-4cbb-a4bf-dc77dd9038a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:17.906342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5793e-ccec-4cbb-a4bf-dc77dd9038a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:17.914881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3da36-229e-4a94-a513-de48b59a268a None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:17.916170 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5793e-ccec-4cbb-a4bf-dc77dd9038a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:17.917684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5793e-ccec-4cbb-a4bf-dc77dd9038a1 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-32f5793e-ccec-4cbb-a4bf-dc77dd9038a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:26:17.932214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:17.933218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:26:17.934111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:26:17.934111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:26:17.934111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:17.934111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 534/1097] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:26:17.939883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5793e-ccec-4cbb-a4bf-dc77dd9038a1 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:17.949905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56fc18-44cf-4173-90a5-e84af4965a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:17.950516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56fc18-44cf-4173-90a5-e84af4965a7b None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:17.950799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56fc18-44cf-4173-90a5-e84af4965a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:17.951072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56fc18-44cf-4173-90a5-e84af4965a7b None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:17.951744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56fc18-44cf-4173-90a5-e84af4965a7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:17.952104 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56fc18-44cf-4173-90a5-e84af4965a7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:17.953995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56fc18-44cf-4173-90a5-e84af4965a7b None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:17.962370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 564/1098] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:26:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:17.968576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56fc18-44cf-4173-90a5-e84af4965a7b None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:17.999633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:18.044592 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5793e-ccec-4cbb-a4bf-dc77dd9038a1 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:18.073378 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56fc18-44cf-4173-90a5-e84af4965a7b None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:18.079400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba71412-2a6b-4dcb-878d-fb9c96ce7455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:18.079999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba71412-2a6b-4dcb-878d-fb9c96ce7455 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:18.080270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba71412-2a6b-4dcb-878d-fb9c96ce7455 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:18.080679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba71412-2a6b-4dcb-878d-fb9c96ce7455 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:18.081194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba71412-2a6b-4dcb-878d-fb9c96ce7455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:18.081649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba71412-2a6b-4dcb-878d-fb9c96ce7455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:18.082834 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba71412-2a6b-4dcb-878d-fb9c96ce7455 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:18.085206 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:18.095718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 535/1099] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:18.101412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba71412-2a6b-4dcb-878d-fb9c96ce7455 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:18.107414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2090b7f-17cb-4028-b565-9bf82c1b9697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:18.108061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2090b7f-17cb-4028-b565-9bf82c1b9697 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:18.108851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2090b7f-17cb-4028-b565-9bf82c1b9697 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:18.109193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2090b7f-17cb-4028-b565-9bf82c1b9697 None admin] PATH_INFO: `/v3/users/f4fb9d7f53a048548e4b09618fb3887d` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:18.110444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2090b7f-17cb-4028-b565-9bf82c1b9697 None admin] RBAC: Authorizing `identity:get_user(user_id=f4fb9d7f53a048548e4b09618fb3887d)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:18.111912 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2090b7f-17cb-4028-b565-9bf82c1b9697 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:18.116196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 565/1100] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:26:17 2022] GET /identity/v3/users/f4fb9d7f53a048548e4b09618fb3887d => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:18.119360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2090b7f-17cb-4028-b565-9bf82c1b9697 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:18.204321 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba71412-2a6b-4dcb-878d-fb9c96ce7455 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:18.214704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:18.219552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2090b7f-17cb-4028-b565-9bf82c1b9697 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:18.238520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8b1e2-19fc-4b43-a7ed-cb687adfd819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:18.239146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8b1e2-19fc-4b43-a7ed-cb687adfd819 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:18.239428 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8b1e2-19fc-4b43-a7ed-cb687adfd819 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:18.239701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8b1e2-19fc-4b43-a7ed-cb687adfd819 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:18.246245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8b1e2-19fc-4b43-a7ed-cb687adfd819 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:18.247689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8b1e2-19fc-4b43-a7ed-cb687adfd819 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:18.254134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4565e-c147-40e8-9676-53b75b422811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:18.254746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4565e-c147-40e8-9676-53b75b422811 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:18.255029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4565e-c147-40e8-9676-53b75b422811 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:18.255301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4565e-c147-40e8-9676-53b75b422811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-47c8b1e2-19fc-4b43-a7ed-cb687adfd819 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:26:18.258974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:18.259910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:18.260831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:26:18.260831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:26:18.260831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:26:18.260831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:26:18.260831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:26:18.260831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:26:18.260831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:26:18.260831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:18.260831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 536/1101] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:26:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:26:18.266459 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8b1e2-19fc-4b43-a7ed-cb687adfd819 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:18.321595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:18.323384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4565e-c147-40e8-9676-53b75b422811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:18.324909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a4565e-c147-40e8-9676-53b75b422811 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:18.367894 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8b1e2-19fc-4b43-a7ed-cb687adfd819 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:18.401728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012c24b-ec1d-470c-a12f-962149f72e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:18.402634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c24b-ec1d-470c-a12f-962149f72e71 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:18.402906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c24b-ec1d-470c-a12f-962149f72e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:18.403175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c24b-ec1d-470c-a12f-962149f72e71 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:18.403928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012c24b-ec1d-470c-a12f-962149f72e71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:18.404220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012c24b-ec1d-470c-a12f-962149f72e71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:18.406246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012c24b-ec1d-470c-a12f-962149f72e71 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:18.422981 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:18.423655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 537/1102] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:26:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:18.429743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012c24b-ec1d-470c-a12f-962149f72e71 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:18.432961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 566/1103] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:26:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 08:26:18.438873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4565e-c147-40e8-9676-53b75b422811 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:18.536179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012c24b-ec1d-470c-a12f-962149f72e71 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:18.541492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4565e-c147-40e8-9676-53b75b422811 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:18.553592 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:18.574461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da35264-0c60-4080-8347-446961c6206e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:18.575071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da35264-0c60-4080-8347-446961c6206e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:18.575344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da35264-0c60-4080-8347-446961c6206e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:18.575611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da35264-0c60-4080-8347-446961c6206e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:18.576329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da35264-0c60-4080-8347-446961c6206e None admin] RBAC: Adding query filter params (role.id=323909a5cc584ec1957e40e8928f214f, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=9c3c6efc3c154ff89066035cd9583459) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:18.576633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da35264-0c60-4080-8347-446961c6206e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:18.577666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb835f1c-8d1f-4633-a556-5efe25e23d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:18.578088 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da35264-0c60-4080-8347-446961c6206e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:18.578297 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb835f1c-8d1f-4633-a556-5efe25e23d6b None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:18.578575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb835f1c-8d1f-4633-a556-5efe25e23d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:18.578847 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb835f1c-8d1f-4633-a556-5efe25e23d6b None admin] PATH_INFO: `/v3/projects/e37f1342ee5f476e8e769844e4f228a6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:18.580027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb835f1c-8d1f-4633-a556-5efe25e23d6b None admin] RBAC: Authorizing `identity:get_project(project_id=e37f1342ee5f476e8e769844e4f228a6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:18.581452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb835f1c-8d1f-4633-a556-5efe25e23d6b None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:18.583107 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 567/1104] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:26:18 2022] GET /identity/v3/projects/e37f1342ee5f476e8e769844e4f228a6 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:18.588367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 538/1105] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:26:18 2022] GET /identity/v3/role_assignments?user.id=9c3c6efc3c154ff89066035cd9583459&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=323909a5cc584ec1957e40e8928f214f => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:26:18.589395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb835f1c-8d1f-4633-a556-5efe25e23d6b None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:18.661025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:18.703650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb835f1c-8d1f-4633-a556-5efe25e23d6b None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:18.740844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706758b-f1f6-46a7-9b18-bb493b56ef2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:18.741489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706758b-f1f6-46a7-9b18-bb493b56ef2f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:18.741766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706758b-f1f6-46a7-9b18-bb493b56ef2f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:18.742036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706758b-f1f6-46a7-9b18-bb493b56ef2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:18.742773 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706758b-f1f6-46a7-9b18-bb493b56ef2f None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=e37f1342ee5f476e8e769844e4f228a6, user.id=f4fb9d7f53a048548e4b09618fb3887d) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:18.743082 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706758b-f1f6-46a7-9b18-bb493b56ef2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:18.744523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706758b-f1f6-46a7-9b18-bb493b56ef2f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:18.751456 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 568/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:26:18 2022] GET /identity/v3/role_assignments?user.id=f4fb9d7f53a048548e4b09618fb3887d&scope.project.id=e37f1342ee5f476e8e769844e4f228a6&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:20.952834 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b2ce6-e039-435c-a313-3250a01eaca3 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:20.953361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b2ce6-e039-435c-a313-3250a01eaca3 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:20.953361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b2ce6-e039-435c-a313-3250a01eaca3 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:20.954657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 539/1107] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:20.960003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e9f0e-0873-4330-9c0e-1325774440ad None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:20.960381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e9f0e-0873-4330-9c0e-1325774440ad None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:20.960550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e9f0e-0873-4330-9c0e-1325774440ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:20.979596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-694e9f0e-0873-4330-9c0e-1325774440ad None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:20.980793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-694e9f0e-0873-4330-9c0e-1325774440ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:21.046811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 569/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:21.318706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b2ce6-e039-435c-a313-3250a01eaca3 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:21.426811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b2ce6-e039-435c-a313-3250a01eaca3 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:21.462489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba58344-f5ae-4cd0-83a6-55f66f9bcb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:21.463117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba58344-f5ae-4cd0-83a6-55f66f9bcb2a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:21.463396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba58344-f5ae-4cd0-83a6-55f66f9bcb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:21.463665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba58344-f5ae-4cd0-83a6-55f66f9bcb2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:21.464785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba58344-f5ae-4cd0-83a6-55f66f9bcb2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:21.466369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba58344-f5ae-4cd0-83a6-55f66f9bcb2a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:21.467985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 540/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:21.473992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e9f0e-0873-4330-9c0e-1325774440ad None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:21.541274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:21.584201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e9f0e-0873-4330-9c0e-1325774440ad None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:21.618923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f133c4f-1d67-44be-843b-817ca75e3017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:21.619790 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f133c4f-1d67-44be-843b-817ca75e3017 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:21.620347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f133c4f-1d67-44be-843b-817ca75e3017 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:21.620844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f133c4f-1d67-44be-843b-817ca75e3017 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:21.621713 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f133c4f-1d67-44be-843b-817ca75e3017 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:21.623148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f133c4f-1d67-44be-843b-817ca75e3017 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:21.648934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 570/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:26:21 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:26:23.875493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255532bb-f46b-4c9d-926f-30d30607877d None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:23.876015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255532bb-f46b-4c9d-926f-30d30607877d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:23.876015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255532bb-f46b-4c9d-926f-30d30607877d None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:23.877287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 541/1111] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:23.882662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d2d0-e660-47d4-827e-0714906c22cb None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:23.883051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d2d0-e660-47d4-827e-0714906c22cb None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:23.883220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d2d0-e660-47d4-827e-0714906c22cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:23.901664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-d0f7d2d0-e660-47d4-827e-0714906c22cb None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:23.903120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d0f7d2d0-e660-47d4-827e-0714906c22cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:23.969554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 571/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:24.246097 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255532bb-f46b-4c9d-926f-30d30607877d None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:24.296271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:24.347350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255532bb-f46b-4c9d-926f-30d30607877d None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:24.380003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f42ff-da7d-4b3c-b2db-2615d14d51cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:24.380623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f42ff-da7d-4b3c-b2db-2615d14d51cb None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:24.380905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f42ff-da7d-4b3c-b2db-2615d14d51cb None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:24.381210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f42ff-da7d-4b3c-b2db-2615d14d51cb None admin] PATH_INFO: `/v3/roles/89694ebde7824b8190bd5b15ece2b11e` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:24.382592 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f42ff-da7d-4b3c-b2db-2615d14d51cb None admin] RBAC: Authorizing `identity:get_role(role_id=89694ebde7824b8190bd5b15ece2b11e)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:24.383941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f42ff-da7d-4b3c-b2db-2615d14d51cb None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:24.385299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 542/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:26:24 2022] GET /identity/v3/roles/89694ebde7824b8190bd5b15ece2b11e => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:24.391790 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7d2d0-e660-47d4-827e-0714906c22cb None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:24.437466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:24.489809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7d2d0-e660-47d4-827e-0714906c22cb None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:24.523333 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb13a-70e2-4d8d-b368-864844548f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:24.523894 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb13a-70e2-4d8d-b368-864844548f61 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:24.524171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb13a-70e2-4d8d-b368-864844548f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:24.524442 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb13a-70e2-4d8d-b368-864844548f61 None admin] PATH_INFO: `/v3/users/959e9331879943fc969f0f517165f902` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:24.538287 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bb13a-70e2-4d8d-b368-864844548f61 None admin] RBAC: Authorizing `identity:get_user(user_id=959e9331879943fc969f0f517165f902)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:24.539703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bb13a-70e2-4d8d-b368-864844548f61 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:24.541333 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 572/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:26:24 2022] GET /identity/v3/users/959e9331879943fc969f0f517165f902 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:24.547703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f42ff-da7d-4b3c-b2db-2615d14d51cb None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:24.648687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f42ff-da7d-4b3c-b2db-2615d14d51cb None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:24.665566 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:24.683740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20a6a4-7122-460f-9226-9ccc9abbad35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:24.684340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20a6a4-7122-460f-9226-9ccc9abbad35 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:24.684620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20a6a4-7122-460f-9226-9ccc9abbad35 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:24.684893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20a6a4-7122-460f-9226-9ccc9abbad35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:24.752927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20a6a4-7122-460f-9226-9ccc9abbad35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:24.756668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20a6a4-7122-460f-9226-9ccc9abbad35 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:24.855190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 543/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:26:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:26:24.861137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb13a-70e2-4d8d-b368-864844548f61 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:24.957756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb13a-70e2-4d8d-b368-864844548f61 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:24.959995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:24.990830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa303672-2d4f-4f63-9133-9807c7c0624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:24.991448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa303672-2d4f-4f63-9133-9807c7c0624f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:24.991728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa303672-2d4f-4f63-9133-9807c7c0624f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:24.992001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa303672-2d4f-4f63-9133-9807c7c0624f None admin] PATH_INFO: `/v3/projects/e37f1342ee5f476e8e769844e4f228a6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:24.993125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa303672-2d4f-4f63-9133-9807c7c0624f None admin] RBAC: Authorizing `identity:get_project(project_id=e37f1342ee5f476e8e769844e4f228a6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:24.994538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa303672-2d4f-4f63-9133-9807c7c0624f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:24.996153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 573/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:26:24 2022] GET /identity/v3/projects/e37f1342ee5f476e8e769844e4f228a6 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:25.002225 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20a6a4-7122-460f-9226-9ccc9abbad35 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:25.019518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:25.103609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20a6a4-7122-460f-9226-9ccc9abbad35 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:25.138695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c26939-b6c7-4b24-956f-3d5c3a6afabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:25.139216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c26939-b6c7-4b24-956f-3d5c3a6afabd None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:25.140023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c26939-b6c7-4b24-956f-3d5c3a6afabd None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:25.140291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c26939-b6c7-4b24-956f-3d5c3a6afabd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:25.141013 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c26939-b6c7-4b24-956f-3d5c3a6afabd None admin] RBAC: Adding query filter params (role.id=89694ebde7824b8190bd5b15ece2b11e, scope.project.id=e37f1342ee5f476e8e769844e4f228a6, user.id=959e9331879943fc969f0f517165f902) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:25.141352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c26939-b6c7-4b24-956f-3d5c3a6afabd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:25.142779 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c26939-b6c7-4b24-956f-3d5c3a6afabd None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:25.148315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 544/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:26:25 2022] GET /identity/v3/role_assignments?user.id=959e9331879943fc969f0f517165f902&scope.project.id=e37f1342ee5f476e8e769844e4f228a6&role.id=89694ebde7824b8190bd5b15ece2b11e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:27.308143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09771b8e-d928-4c5e-9e10-e6ba245e36c1 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:27.308677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09771b8e-d928-4c5e-9e10-e6ba245e36c1 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:27.308677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09771b8e-d928-4c5e-9e10-e6ba245e36c1 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:27.310069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 574/1118] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:27.315975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a6085-20ba-48e6-aaca-c5416325272b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:27.316418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a6085-20ba-48e6-aaca-c5416325272b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:27.316520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a6085-20ba-48e6-aaca-c5416325272b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:27.334312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-1b7a6085-20ba-48e6-aaca-c5416325272b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:27.335495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1b7a6085-20ba-48e6-aaca-c5416325272b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:27.401571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 545/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:27.673671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09771b8e-d928-4c5e-9e10-e6ba245e36c1 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:27.793076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09771b8e-d928-4c5e-9e10-e6ba245e36c1 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:27.829593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165211d-862e-440a-bde5-4a9d2e7fed42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:27.830225 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165211d-862e-440a-bde5-4a9d2e7fed42 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:27.830508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165211d-862e-440a-bde5-4a9d2e7fed42 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:27.830770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165211d-862e-440a-bde5-4a9d2e7fed42 None admin] PATH_INFO: `/v3/roles/89694ebde7824b8190bd5b15ece2b11e` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:27.832157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165211d-862e-440a-bde5-4a9d2e7fed42 None admin] RBAC: Authorizing `identity:get_role(role_id=89694ebde7824b8190bd5b15ece2b11e)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:27.833542 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165211d-862e-440a-bde5-4a9d2e7fed42 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:27.834868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 575/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:26:27 2022] GET /identity/v3/roles/89694ebde7824b8190bd5b15ece2b11e => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:27.840856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a6085-20ba-48e6-aaca-c5416325272b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:27.852878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:27.938322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a6085-20ba-48e6-aaca-c5416325272b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:27.972081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9264004-763d-4c64-ad5f-7dac67c6de8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:27.972701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9264004-763d-4c64-ad5f-7dac67c6de8e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:27.972981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9264004-763d-4c64-ad5f-7dac67c6de8e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:27.973282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9264004-763d-4c64-ad5f-7dac67c6de8e None admin] PATH_INFO: `/v3/users/959e9331879943fc969f0f517165f902` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:27.974390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9264004-763d-4c64-ad5f-7dac67c6de8e None admin] RBAC: Authorizing `identity:get_user(user_id=959e9331879943fc969f0f517165f902)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:27.975805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9264004-763d-4c64-ad5f-7dac67c6de8e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:27.977539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 546/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:26:27 2022] GET /identity/v3/users/959e9331879943fc969f0f517165f902 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:27.983419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165211d-862e-440a-bde5-4a9d2e7fed42 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:28.024999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:28.085252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165211d-862e-440a-bde5-4a9d2e7fed42 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:28.127809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0944f-da18-45c0-ae08-8bafcf249461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:28.128452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0944f-da18-45c0-ae08-8bafcf249461 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:28.128728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0944f-da18-45c0-ae08-8bafcf249461 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:28.129002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0944f-da18-45c0-ae08-8bafcf249461 None admin] PATH_INFO: `/v3/projects/e37f1342ee5f476e8e769844e4f228a6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:28.130157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0944f-da18-45c0-ae08-8bafcf249461 None admin] RBAC: Authorizing `identity:get_project(project_id=e37f1342ee5f476e8e769844e4f228a6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:28.131584 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0944f-da18-45c0-ae08-8bafcf249461 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:28.133251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 576/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:26:27 2022] GET /identity/v3/projects/e37f1342ee5f476e8e769844e4f228a6 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:28.139735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9264004-763d-4c64-ad5f-7dac67c6de8e None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:28.216526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:28.239470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9264004-763d-4c64-ad5f-7dac67c6de8e None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:28.271924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ac975-bdb5-40e9-b7b8-58cff54c8842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:28.272571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ac975-bdb5-40e9-b7b8-58cff54c8842 None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:28.272852 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ac975-bdb5-40e9-b7b8-58cff54c8842 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:28.273151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ac975-bdb5-40e9-b7b8-58cff54c8842 None admin] PATH_INFO: `/v3/projects/e37f1342ee5f476e8e769844e4f228a6/users/959e9331879943fc969f0f517165f902/roles/89694ebde7824b8190bd5b15ece2b11e` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:28.275267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ac975-bdb5-40e9-b7b8-58cff54c8842 None admin] RBAC: Authorizing `identity:create_grant(project_id=e37f1342ee5f476e8e769844e4f228a6, user_id=959e9331879943fc969f0f517165f902, role_id=89694ebde7824b8190bd5b15ece2b11e)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:28.276694 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ac975-bdb5-40e9-b7b8-58cff54c8842 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:28.290364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 547/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:26:28 2022] PUT /identity/v3/projects/e37f1342ee5f476e8e769844e4f228a6/users/959e9331879943fc969f0f517165f902/roles/89694ebde7824b8190bd5b15ece2b11e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:26:30.426900 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69231712-6415-486a-aad3-f00789958822 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:30.427437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69231712-6415-486a-aad3-f00789958822 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:30.427437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69231712-6415-486a-aad3-f00789958822 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:30.428769 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 577/1124] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:30.434022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b097f-583d-4238-840a-4d8080c06419 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:30.434392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b097f-583d-4238-840a-4d8080c06419 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:30.434568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b097f-583d-4238-840a-4d8080c06419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:30.453460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-f30b097f-583d-4238-840a-4d8080c06419 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:30.454646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f30b097f-583d-4238-840a-4d8080c06419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:30.522340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 548/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:30.788557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69231712-6415-486a-aad3-f00789958822 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:30.892547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69231712-6415-486a-aad3-f00789958822 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:30.901444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:30.926204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271ac39-f1f3-40bf-b25a-19011193126f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:30.926822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271ac39-f1f3-40bf-b25a-19011193126f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:30.927099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271ac39-f1f3-40bf-b25a-19011193126f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:30.927371 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271ac39-f1f3-40bf-b25a-19011193126f None admin] PATH_INFO: `/v3/roles/89694ebde7824b8190bd5b15ece2b11e` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:30.928743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271ac39-f1f3-40bf-b25a-19011193126f None admin] RBAC: Authorizing `identity:get_role(role_id=89694ebde7824b8190bd5b15ece2b11e)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:30.930119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271ac39-f1f3-40bf-b25a-19011193126f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:30.931433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 578/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:26:30 2022] GET /identity/v3/roles/89694ebde7824b8190bd5b15ece2b11e => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:30.937429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b097f-583d-4238-840a-4d8080c06419 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:31.001977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:31.037357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b097f-583d-4238-840a-4d8080c06419 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:31.071891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1084a99-62ae-4eda-9366-b1fdf91a5dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:31.072437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1084a99-62ae-4eda-9366-b1fdf91a5dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:31.072718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1084a99-62ae-4eda-9366-b1fdf91a5dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:31.072993 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1084a99-62ae-4eda-9366-b1fdf91a5dc5 None admin] PATH_INFO: `/v3/users/959e9331879943fc969f0f517165f902` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:31.074162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1084a99-62ae-4eda-9366-b1fdf91a5dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=959e9331879943fc969f0f517165f902)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:31.075598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1084a99-62ae-4eda-9366-b1fdf91a5dc5 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:31.077752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 549/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:26:30 2022] GET /identity/v3/users/959e9331879943fc969f0f517165f902 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:31.083583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271ac39-f1f3-40bf-b25a-19011193126f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:31.189811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271ac39-f1f3-40bf-b25a-19011193126f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:31.223305 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86384-a812-44c0-8fc5-45a4f3bc32af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:31.223906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86384-a812-44c0-8fc5-45a4f3bc32af None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:31.224189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86384-a812-44c0-8fc5-45a4f3bc32af None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:31.224464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86384-a812-44c0-8fc5-45a4f3bc32af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:31.269466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:31.292031 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa86384-a812-44c0-8fc5-45a4f3bc32af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:31.293498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa86384-a812-44c0-8fc5-45a4f3bc32af None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:31.395775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 579/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:26:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:26:31.401853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1084a99-62ae-4eda-9366-b1fdf91a5dc5 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:31.500140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1084a99-62ae-4eda-9366-b1fdf91a5dc5 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:31.534184 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dbd0e-3885-4ff5-82ea-a1513548542e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:31.534812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dbd0e-3885-4ff5-82ea-a1513548542e None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:31.535089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dbd0e-3885-4ff5-82ea-a1513548542e None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:31.535354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dbd0e-3885-4ff5-82ea-a1513548542e None admin] PATH_INFO: `/v3/projects/e37f1342ee5f476e8e769844e4f228a6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:31.536445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230dbd0e-3885-4ff5-82ea-a1513548542e None admin] RBAC: Authorizing `identity:get_project(project_id=e37f1342ee5f476e8e769844e4f228a6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:31.537892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230dbd0e-3885-4ff5-82ea-a1513548542e None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:31.539518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 550/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:26:31 2022] GET /identity/v3/projects/e37f1342ee5f476e8e769844e4f228a6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:31.545598 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86384-a812-44c0-8fc5-45a4f3bc32af None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:31.621496 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:31.645048 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86384-a812-44c0-8fc5-45a4f3bc32af None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:31.678824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3e781-84a3-4dbc-93b3-e0e263f906f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:31.679357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3e781-84a3-4dbc-93b3-e0e263f906f1 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:31.679636 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3e781-84a3-4dbc-93b3-e0e263f906f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:31.679907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3e781-84a3-4dbc-93b3-e0e263f906f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:31.680632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3e781-84a3-4dbc-93b3-e0e263f906f1 None admin] RBAC: Adding query filter params (role.id=89694ebde7824b8190bd5b15ece2b11e, scope.project.id=e37f1342ee5f476e8e769844e4f228a6, user.id=959e9331879943fc969f0f517165f902) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:31.680940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3e781-84a3-4dbc-93b3-e0e263f906f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:31.682385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3e781-84a3-4dbc-93b3-e0e263f906f1 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:31.688539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 580/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:26:31 2022] GET /identity/v3/role_assignments?user.id=959e9331879943fc969f0f517165f902&scope.project.id=e37f1342ee5f476e8e769844e4f228a6&role.id=89694ebde7824b8190bd5b15ece2b11e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:33.878860 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40c6ed-3470-4e2b-87d9-a5a01464ca91 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:33.879362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40c6ed-3470-4e2b-87d9-a5a01464ca91 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:33.879539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40c6ed-3470-4e2b-87d9-a5a01464ca91 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:33.881652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 551/1131] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:33.887792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d305f4-5892-424f-b585-a547d49c5aae None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:33.888184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d305f4-5892-424f-b585-a547d49c5aae None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:33.888581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d305f4-5892-424f-b585-a547d49c5aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:33.909634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-57d305f4-5892-424f-b585-a547d49c5aae None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:33.910823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-57d305f4-5892-424f-b585-a547d49c5aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:33.975969 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 581/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:34.253877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40c6ed-3470-4e2b-87d9-a5a01464ca91 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:34.257740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:34.352471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40c6ed-3470-4e2b-87d9-a5a01464ca91 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:34.385428 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42154e77-f8cf-49f2-9006-aa5d487e02ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:34.386054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42154e77-f8cf-49f2-9006-aa5d487e02ab None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:34.386331 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42154e77-f8cf-49f2-9006-aa5d487e02ab None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:34.386603 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42154e77-f8cf-49f2-9006-aa5d487e02ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:34.387707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42154e77-f8cf-49f2-9006-aa5d487e02ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:34.389172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42154e77-f8cf-49f2-9006-aa5d487e02ab None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:34.390733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 552/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:34.396596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d305f4-5892-424f-b585-a547d49c5aae None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:34.463585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:34.494863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d305f4-5892-424f-b585-a547d49c5aae None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:34.532658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b8931-e1ea-4c34-b7b4-41646f55f6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:34.533312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8931-e1ea-4c34-b7b4-41646f55f6de None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:34.533595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8931-e1ea-4c34-b7b4-41646f55f6de None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:34.533868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8931-e1ea-4c34-b7b4-41646f55f6de None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:34.534528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b8931-e1ea-4c34-b7b4-41646f55f6de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:34.535956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b8931-e1ea-4c34-b7b4-41646f55f6de None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:34.564526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 582/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:26:34 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:26:36.758565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77123a6e-d062-4054-8b6e-ca1d0b124cb0 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:36.759121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77123a6e-d062-4054-8b6e-ca1d0b124cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:36.759121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77123a6e-d062-4054-8b6e-ca1d0b124cb0 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:36.760456 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 553/1135] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:36.766164 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553c1c0-c99f-4b65-b9e2-d40b00d2e816 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:36.766569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553c1c0-c99f-4b65-b9e2-d40b00d2e816 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:36.766705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553c1c0-c99f-4b65-b9e2-d40b00d2e816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:36.784723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-a553c1c0-c99f-4b65-b9e2-d40b00d2e816 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:36.785865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a553c1c0-c99f-4b65-b9e2-d40b00d2e816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:36.858701 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 583/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:37.138118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77123a6e-d062-4054-8b6e-ca1d0b124cb0 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:37.209280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:37.240845 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77123a6e-d062-4054-8b6e-ca1d0b124cb0 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:37.273886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30568bc1-9455-48be-9d71-5c35efd06d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:37.274515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30568bc1-9455-48be-9d71-5c35efd06d74 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:37.274790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30568bc1-9455-48be-9d71-5c35efd06d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:37.275189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30568bc1-9455-48be-9d71-5c35efd06d74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:37.276283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30568bc1-9455-48be-9d71-5c35efd06d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:37.277987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30568bc1-9455-48be-9d71-5c35efd06d74 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:37.279551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 554/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:26:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:37.285943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553c1c0-c99f-4b65-b9e2-d40b00d2e816 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:37.336021 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:37.385596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553c1c0-c99f-4b65-b9e2-d40b00d2e816 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:37.420343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b229cd-92b6-444a-8c48-b9bad5aee2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:37.421439 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b229cd-92b6-444a-8c48-b9bad5aee2b4 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:37.421714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b229cd-92b6-444a-8c48-b9bad5aee2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:37.421982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b229cd-92b6-444a-8c48-b9bad5aee2b4 None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:37.422639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b229cd-92b6-444a-8c48-b9bad5aee2b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:37.424157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b229cd-92b6-444a-8c48-b9bad5aee2b4 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:37.444675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 584/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:26:37 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:26:39.650429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be61d18-d7db-4c6d-85ea-5c218e0f5c6f None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:39.651022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be61d18-d7db-4c6d-85ea-5c218e0f5c6f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:39.651022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be61d18-d7db-4c6d-85ea-5c218e0f5c6f None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:39.652354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 555/1139] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:39.658991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e18255-ceac-4bc4-956b-ce10121189cd None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:39.659464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e18255-ceac-4bc4-956b-ce10121189cd None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:39.659557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e18255-ceac-4bc4-956b-ce10121189cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:39.678236 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-00e18255-ceac-4bc4-956b-ce10121189cd None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:39.679622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-00e18255-ceac-4bc4-956b-ce10121189cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:39.749070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 585/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:40.021969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be61d18-d7db-4c6d-85ea-5c218e0f5c6f None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:40.126638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be61d18-d7db-4c6d-85ea-5c218e0f5c6f None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:40.165477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11494f51-c791-42ca-9b8d-6b05d0017fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:40.166121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11494f51-c791-42ca-9b8d-6b05d0017fca None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:40.166400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11494f51-c791-42ca-9b8d-6b05d0017fca None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:40.166675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11494f51-c791-42ca-9b8d-6b05d0017fca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:40.175103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:40.178418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11494f51-c791-42ca-9b8d-6b05d0017fca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-11494f51-c791-42ca-9b8d-6b05d0017fca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:26:40.183264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:40.184446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:40.185684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 556/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:26:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:40.191723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e18255-ceac-4bc4-956b-ce10121189cd None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:40.232454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:40.294351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e18255-ceac-4bc4-956b-ce10121189cd None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:40.327865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cf7ec-5672-4142-b4e1-255bd1a263d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:40.328467 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cf7ec-5672-4142-b4e1-255bd1a263d8 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:40.328747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cf7ec-5672-4142-b4e1-255bd1a263d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:40.329019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cf7ec-5672-4142-b4e1-255bd1a263d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:40.329788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cf7ec-5672-4142-b4e1-255bd1a263d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:40.330091 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cf7ec-5672-4142-b4e1-255bd1a263d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:40.331441 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cf7ec-5672-4142-b4e1-255bd1a263d8 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:40.339340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 586/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:26:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:40.345903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11494f51-c791-42ca-9b8d-6b05d0017fca None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:40.444404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11494f51-c791-42ca-9b8d-6b05d0017fca None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:40.478606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f73eb8-839f-4d84-a60a-68fe70cad049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:40.481229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f73eb8-839f-4d84-a60a-68fe70cad049 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:40.481516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f73eb8-839f-4d84-a60a-68fe70cad049 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:40.481793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f73eb8-839f-4d84-a60a-68fe70cad049 None admin] PATH_INFO: `/v3/users/02db24c2ef9d43d39c44ab768efda751` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:40.498559 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f73eb8-839f-4d84-a60a-68fe70cad049 None admin] RBAC: Authorizing `identity:get_user(user_id=02db24c2ef9d43d39c44ab768efda751)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:40.500071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f73eb8-839f-4d84-a60a-68fe70cad049 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:40.501785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 557/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:26:40 2022] GET /identity/v3/users/02db24c2ef9d43d39c44ab768efda751 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:40.507837 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cf7ec-5672-4142-b4e1-255bd1a263d8 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:40.617563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:40.620748 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cf7ec-5672-4142-b4e1-255bd1a263d8 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:40.657602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928bcf5-1aae-43f8-ba14-e16124b53036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:40.658213 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928bcf5-1aae-43f8-ba14-e16124b53036 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:40.658493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928bcf5-1aae-43f8-ba14-e16124b53036 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:40.658766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928bcf5-1aae-43f8-ba14-e16124b53036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:40.729367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928bcf5-1aae-43f8-ba14-e16124b53036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:40.732013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928bcf5-1aae-43f8-ba14-e16124b53036 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:40.835977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:40.840363 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 587/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:26:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:26:40.849968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f73eb8-839f-4d84-a60a-68fe70cad049 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:40.913578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:40.958363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f73eb8-839f-4d84-a60a-68fe70cad049 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:40.991965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732a53f-9195-472b-b9f4-6accfa86e38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:40.992586 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732a53f-9195-472b-b9f4-6accfa86e38a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:40.992858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732a53f-9195-472b-b9f4-6accfa86e38a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:40.993157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732a53f-9195-472b-b9f4-6accfa86e38a None admin] PATH_INFO: `/v3/projects/d2dc5bc7f604461e8086f28ae1f737b7` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:40.994254 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732a53f-9195-472b-b9f4-6accfa86e38a None admin] RBAC: Authorizing `identity:get_project(project_id=d2dc5bc7f604461e8086f28ae1f737b7)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:40.995669 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732a53f-9195-472b-b9f4-6accfa86e38a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:40.997335 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 558/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:26:40 2022] GET /identity/v3/projects/d2dc5bc7f604461e8086f28ae1f737b7 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:41.004104 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928bcf5-1aae-43f8-ba14-e16124b53036 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:41.102758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928bcf5-1aae-43f8-ba14-e16124b53036 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:41.136570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efd695-cb54-4858-b8e1-8904d824d150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:41.137263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efd695-cb54-4858-b8e1-8904d824d150 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:41.137544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efd695-cb54-4858-b8e1-8904d824d150 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:41.137816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efd695-cb54-4858-b8e1-8904d824d150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:41.138543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efd695-cb54-4858-b8e1-8904d824d150 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=d2dc5bc7f604461e8086f28ae1f737b7, user.id=02db24c2ef9d43d39c44ab768efda751) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:41.138854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efd695-cb54-4858-b8e1-8904d824d150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:41.140957 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efd695-cb54-4858-b8e1-8904d824d150 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:41.146237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 588/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:26:41 2022] GET /identity/v3/role_assignments?user.id=02db24c2ef9d43d39c44ab768efda751&scope.project.id=d2dc5bc7f604461e8086f28ae1f737b7&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:43.281282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6e2ec-81af-4eb8-87d2-0925ff2ce03e None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:43.281845 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6e2ec-81af-4eb8-87d2-0925ff2ce03e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:43.281845 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6e2ec-81af-4eb8-87d2-0925ff2ce03e None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:43.283146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 559/1147] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:43.288482 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34d64d-a076-4cb6-80a0-affde2256a24 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:43.288892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34d64d-a076-4cb6-80a0-affde2256a24 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:43.289036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34d64d-a076-4cb6-80a0-affde2256a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:43.308098 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2c34d64d-a076-4cb6-80a0-affde2256a24 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:43.309321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2c34d64d-a076-4cb6-80a0-affde2256a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:43.367992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:43.377235 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 589/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:43.655902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6e2ec-81af-4eb8-87d2-0925ff2ce03e None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:43.754311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d6e2ec-81af-4eb8-87d2-0925ff2ce03e None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:43.779563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:43.789159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7436cf-3808-4f87-872f-78f79a247474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:43.789782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7436cf-3808-4f87-872f-78f79a247474 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:43.790061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7436cf-3808-4f87-872f-78f79a247474 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:43.790329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7436cf-3808-4f87-872f-78f79a247474 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:43.800218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7436cf-3808-4f87-872f-78f79a247474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-3e7436cf-3808-4f87-872f-78f79a247474 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:26:43.804980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:43.806095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:43.807129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 560/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:26:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:43.812239 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34d64d-a076-4cb6-80a0-affde2256a24 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:43.914632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34d64d-a076-4cb6-80a0-affde2256a24 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:43.947374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ce6fd-f6fd-4705-9308-ad9b2444e422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:43.947974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ce6fd-f6fd-4705-9308-ad9b2444e422 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:43.948252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ce6fd-f6fd-4705-9308-ad9b2444e422 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:43.948524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ce6fd-f6fd-4705-9308-ad9b2444e422 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:43.949226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ce6fd-f6fd-4705-9308-ad9b2444e422 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:43.949528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ce6fd-f6fd-4705-9308-ad9b2444e422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:43.950866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ce6fd-f6fd-4705-9308-ad9b2444e422 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:43.959529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 590/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:26:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:43.965429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7436cf-3808-4f87-872f-78f79a247474 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:44.064741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7436cf-3808-4f87-872f-78f79a247474 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:44.098721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6df851-c532-47d9-af64-645c57af863d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:44.099342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6df851-c532-47d9-af64-645c57af863d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:44.099617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6df851-c532-47d9-af64-645c57af863d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:44.099890 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6df851-c532-47d9-af64-645c57af863d None admin] PATH_INFO: `/v3/users/02db24c2ef9d43d39c44ab768efda751` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:44.101000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6df851-c532-47d9-af64-645c57af863d None admin] RBAC: Authorizing `identity:get_user(user_id=02db24c2ef9d43d39c44ab768efda751)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:44.102443 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6df851-c532-47d9-af64-645c57af863d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:44.104006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 561/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:26:43 2022] GET /identity/v3/users/02db24c2ef9d43d39c44ab768efda751 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:44.109912 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ce6fd-f6fd-4705-9308-ad9b2444e422 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:44.165359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:44.208972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ce6fd-f6fd-4705-9308-ad9b2444e422 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:44.240632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c863f6-804a-4314-9349-f3eab47a08a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:44.241282 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c863f6-804a-4314-9349-f3eab47a08a5 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:44.241569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c863f6-804a-4314-9349-f3eab47a08a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:44.241840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c863f6-804a-4314-9349-f3eab47a08a5 None admin] PATH_INFO: `/v3/projects/d2dc5bc7f604461e8086f28ae1f737b7` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:44.242946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c863f6-804a-4314-9349-f3eab47a08a5 None admin] RBAC: Authorizing `identity:get_project(project_id=d2dc5bc7f604461e8086f28ae1f737b7)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:44.244351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c863f6-804a-4314-9349-f3eab47a08a5 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:44.245995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 591/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:26:44 2022] GET /identity/v3/projects/d2dc5bc7f604461e8086f28ae1f737b7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:44.252496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6df851-c532-47d9-af64-645c57af863d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:44.307306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:44.351413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6df851-c532-47d9-af64-645c57af863d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:44.385297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddaa42-3626-4643-a010-7708d58792f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:44.385957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddaa42-3626-4643-a010-7708d58792f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:44.386238 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddaa42-3626-4643-a010-7708d58792f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:44.386538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddaa42-3626-4643-a010-7708d58792f7 None admin] PATH_INFO: `/v3/projects/d2dc5bc7f604461e8086f28ae1f737b7/users/02db24c2ef9d43d39c44ab768efda751/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:44.388437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddaa42-3626-4643-a010-7708d58792f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2dc5bc7f604461e8086f28ae1f737b7, user_id=02db24c2ef9d43d39c44ab768efda751, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:44.390041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddaa42-3626-4643-a010-7708d58792f7 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:44.403118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 562/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:26:44 2022] PUT /identity/v3/projects/d2dc5bc7f604461e8086f28ae1f737b7/users/02db24c2ef9d43d39c44ab768efda751/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:26:46.558034 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10c8bd-cbab-49ea-8522-0b87fc13b9d4 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:46.558558 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10c8bd-cbab-49ea-8522-0b87fc13b9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:46.558670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10c8bd-cbab-49ea-8522-0b87fc13b9d4 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:46.560320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 592/1154] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:46.566151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25a930-2fe9-408c-bb9f-583e82385113 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:46.566655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25a930-2fe9-408c-bb9f-583e82385113 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:46.566746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25a930-2fe9-408c-bb9f-583e82385113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:46.585323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-7a25a930-2fe9-408c-bb9f-583e82385113 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:46.586521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7a25a930-2fe9-408c-bb9f-583e82385113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:46.654303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 563/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:46.923302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10c8bd-cbab-49ea-8522-0b87fc13b9d4 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:47.023755 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10c8bd-cbab-49ea-8522-0b87fc13b9d4 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:47.047461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:47.058237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66793835-1a4b-4f8e-9c94-ed4109d25bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:47.058860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66793835-1a4b-4f8e-9c94-ed4109d25bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:47.059140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66793835-1a4b-4f8e-9c94-ed4109d25bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:47.059411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66793835-1a4b-4f8e-9c94-ed4109d25bf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:47.069300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66793835-1a4b-4f8e-9c94-ed4109d25bf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-66793835-1a4b-4f8e-9c94-ed4109d25bf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:26:47.074077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:47.075182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:26:47.076136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 593/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:26:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:47.081521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25a930-2fe9-408c-bb9f-583e82385113 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:47.127883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:47.181273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25a930-2fe9-408c-bb9f-583e82385113 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:47.215590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f88d2-f801-41db-b6d5-b38c1ebb8b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:47.216188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f88d2-f801-41db-b6d5-b38c1ebb8b77 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:47.216464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f88d2-f801-41db-b6d5-b38c1ebb8b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:47.216734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f88d2-f801-41db-b6d5-b38c1ebb8b77 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:47.217932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f88d2-f801-41db-b6d5-b38c1ebb8b77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:47.218232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f88d2-f801-41db-b6d5-b38c1ebb8b77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:47.219584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f88d2-f801-41db-b6d5-b38c1ebb8b77 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:47.227392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 564/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:26:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:47.234170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66793835-1a4b-4f8e-9c94-ed4109d25bf9 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:47.331089 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66793835-1a4b-4f8e-9c94-ed4109d25bf9 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:47.363957 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ca35b-907e-4735-b4d8-31cae6463d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:47.364576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ca35b-907e-4735-b4d8-31cae6463d1f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:47.364853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ca35b-907e-4735-b4d8-31cae6463d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:47.365150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ca35b-907e-4735-b4d8-31cae6463d1f None admin] PATH_INFO: `/v3/users/02db24c2ef9d43d39c44ab768efda751` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:47.366252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ca35b-907e-4735-b4d8-31cae6463d1f None admin] RBAC: Authorizing `identity:get_user(user_id=02db24c2ef9d43d39c44ab768efda751)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:47.367669 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ca35b-907e-4735-b4d8-31cae6463d1f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:47.369313 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 594/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:26:47 2022] GET /identity/v3/users/02db24c2ef9d43d39c44ab768efda751 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:47.375857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f88d2-f801-41db-b6d5-b38c1ebb8b77 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:47.478910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f88d2-f801-41db-b6d5-b38c1ebb8b77 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:47.486225 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:47.513491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003a27f-488f-4ca0-8f0d-f7ba3cffb21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:47.514128 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003a27f-488f-4ca0-8f0d-f7ba3cffb21b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:47.514408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003a27f-488f-4ca0-8f0d-f7ba3cffb21b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:47.514679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003a27f-488f-4ca0-8f0d-f7ba3cffb21b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:47.580689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003a27f-488f-4ca0-8f0d-f7ba3cffb21b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:47.582159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003a27f-488f-4ca0-8f0d-f7ba3cffb21b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:47.679132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 565/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:26:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:26:47.685181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ca35b-907e-4735-b4d8-31cae6463d1f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:47.738025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:47.785054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ca35b-907e-4735-b4d8-31cae6463d1f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:47.817833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0989d4-6833-42c0-8308-37d601cd3e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:47.818525 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0989d4-6833-42c0-8308-37d601cd3e0b None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:47.818736 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0989d4-6833-42c0-8308-37d601cd3e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:47.819007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0989d4-6833-42c0-8308-37d601cd3e0b None admin] PATH_INFO: `/v3/projects/d2dc5bc7f604461e8086f28ae1f737b7` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:47.820140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0989d4-6833-42c0-8308-37d601cd3e0b None admin] RBAC: Authorizing `identity:get_project(project_id=d2dc5bc7f604461e8086f28ae1f737b7)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:47.821601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0989d4-6833-42c0-8308-37d601cd3e0b None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:47.823267 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 595/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:26:47 2022] GET /identity/v3/projects/d2dc5bc7f604461e8086f28ae1f737b7 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:47.830085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003a27f-488f-4ca0-8f0d-f7ba3cffb21b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:47.838148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:47.926395 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003a27f-488f-4ca0-8f0d-f7ba3cffb21b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:47.960284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa1aa-4905-4c88-adb3-35a394bf8765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:47.960881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa1aa-4905-4c88-adb3-35a394bf8765 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:47.961332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa1aa-4905-4c88-adb3-35a394bf8765 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:47.961607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa1aa-4905-4c88-adb3-35a394bf8765 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:47.962318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa1aa-4905-4c88-adb3-35a394bf8765 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=d2dc5bc7f604461e8086f28ae1f737b7, user.id=02db24c2ef9d43d39c44ab768efda751) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:47.962624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa1aa-4905-4c88-adb3-35a394bf8765 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:47.964044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa1aa-4905-4c88-adb3-35a394bf8765 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:47.969702 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 566/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:26:47 2022] GET /identity/v3/role_assignments?user.id=02db24c2ef9d43d39c44ab768efda751&scope.project.id=d2dc5bc7f604461e8086f28ae1f737b7&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:50.163935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2be1e-aeb1-4691-bff2-b572ffc92690 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:50.164494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2be1e-aeb1-4691-bff2-b572ffc92690 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:50.164494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2be1e-aeb1-4691-bff2-b572ffc92690 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:50.165835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 596/1162] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:50.171354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec576a2-58b4-4b45-8e3f-ee0ed49947cd None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:50.171756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec576a2-58b4-4b45-8e3f-ee0ed49947cd None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:50.171892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec576a2-58b4-4b45-8e3f-ee0ed49947cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:50.190199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-aec576a2-58b4-4b45-8e3f-ee0ed49947cd None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:50.191364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-aec576a2-58b4-4b45-8e3f-ee0ed49947cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:50.252238 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:50.257450 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 567/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:50.521187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2be1e-aeb1-4691-bff2-b572ffc92690 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:50.620238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2be1e-aeb1-4691-bff2-b572ffc92690 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:50.640124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:50.655737 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab9d72-2767-4eba-81b1-02288ac00654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:50.656434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab9d72-2767-4eba-81b1-02288ac00654 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:50.656802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab9d72-2767-4eba-81b1-02288ac00654 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:50.657221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab9d72-2767-4eba-81b1-02288ac00654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:50.725073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab9d72-2767-4eba-81b1-02288ac00654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:50.726543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab9d72-2767-4eba-81b1-02288ac00654 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:50.822853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 597/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:26:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:26:50.829692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec576a2-58b4-4b45-8e3f-ee0ed49947cd None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:50.933123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec576a2-58b4-4b45-8e3f-ee0ed49947cd None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:50.972299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006a406-09fb-448d-acd3-c7f3bca5818a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:50.972933 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006a406-09fb-448d-acd3-c7f3bca5818a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:50.973249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006a406-09fb-448d-acd3-c7f3bca5818a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:50.973522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006a406-09fb-448d-acd3-c7f3bca5818a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:50.980172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006a406-09fb-448d-acd3-c7f3bca5818a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:50.981635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006a406-09fb-448d-acd3-c7f3bca5818a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-8006a406-09fb-448d-acd3-c7f3bca5818a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:26:50.993458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:26:50.994785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:26:50.995792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 568/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 08:26:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:26:51.000620 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab9d72-2767-4eba-81b1-02288ac00654 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:51.021157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:51.098918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab9d72-2767-4eba-81b1-02288ac00654 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:51.132130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5972492-e8ff-4815-9cd4-5dcf5a501c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:51.132729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5972492-e8ff-4815-9cd4-5dcf5a501c54 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:51.133011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5972492-e8ff-4815-9cd4-5dcf5a501c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:51.133317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5972492-e8ff-4815-9cd4-5dcf5a501c54 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:51.133949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5972492-e8ff-4815-9cd4-5dcf5a501c54 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:26:51.134252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5972492-e8ff-4815-9cd4-5dcf5a501c54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:51.135608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5972492-e8ff-4815-9cd4-5dcf5a501c54 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:51.143238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 598/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 08:26:50 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:53.298216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3f87a-0acf-43f4-84c2-061f554585b2 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:53.298776 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3f87a-0acf-43f4-84c2-061f554585b2 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:53.298776 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3f87a-0acf-43f4-84c2-061f554585b2 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:53.300079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 569/1167] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:53.305336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac65a64-c125-4b69-9c01-f7fa5df026b8 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:53.305767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac65a64-c125-4b69-9c01-f7fa5df026b8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:53.305872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac65a64-c125-4b69-9c01-f7fa5df026b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:53.324201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-aac65a64-c125-4b69-9c01-f7fa5df026b8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:53.325420 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-aac65a64-c125-4b69-9c01-f7fa5df026b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:53.391577 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 599/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:26:53.664106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3f87a-0acf-43f4-84c2-061f554585b2 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:53.723437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:53.764573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3f87a-0acf-43f4-84c2-061f554585b2 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:53.797552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f064b76-3249-40b4-81f9-0884553ffbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:53.798140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f064b76-3249-40b4-81f9-0884553ffbcf None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:53.798415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f064b76-3249-40b4-81f9-0884553ffbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:53.798680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f064b76-3249-40b4-81f9-0884553ffbcf None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:53.799328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f064b76-3249-40b4-81f9-0884553ffbcf None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:53.800675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f064b76-3249-40b4-81f9-0884553ffbcf None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:53.815708 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 570/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Wed Sep 28 08:26:53 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:26:56.029171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff93d79-a183-4366-bde5-43695692dc1d None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:56.030365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff93d79-a183-4366-bde5-43695692dc1d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:56.030365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff93d79-a183-4366-bde5-43695692dc1d None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:56.031640 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 600/1170] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:56.036896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5565663-2a6e-4117-8316-dd187333642c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:56.037347 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5565663-2a6e-4117-8316-dd187333642c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:56.037476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5565663-2a6e-4117-8316-dd187333642c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:56.055250 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-b5565663-2a6e-4117-8316-dd187333642c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:56.056437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b5565663-2a6e-4117-8316-dd187333642c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:56.121784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 571/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:56.389949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff93d79-a183-4366-bde5-43695692dc1d None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:56.410329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:56.494927 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff93d79-a183-4366-bde5-43695692dc1d None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:56.528770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1b116-827c-47f8-a43e-0f3defb04baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:56.529970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1b116-827c-47f8-a43e-0f3defb04baf None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:56.530246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1b116-827c-47f8-a43e-0f3defb04baf None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:56.530517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1b116-827c-47f8-a43e-0f3defb04baf None admin] PATH_INFO: `/v3/domains/5126f54e81364029ac1ae46a51ce6ee7` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:56.540830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1b116-827c-47f8-a43e-0f3defb04baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=5126f54e81364029ac1ae46a51ce6ee7)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:56.542294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1b116-827c-47f8-a43e-0f3defb04baf None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:56.543889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 601/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Sep 28 08:26:56 2022] GET /identity/v3/domains/5126f54e81364029ac1ae46a51ce6ee7 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:56.550336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5565663-2a6e-4117-8316-dd187333642c None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:56.599249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:56.648498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5565663-2a6e-4117-8316-dd187333642c None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:56.682391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11291a-b125-4ad7-b28e-efa3a8ab6068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:56.682992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11291a-b125-4ad7-b28e-efa3a8ab6068 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:56.683270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11291a-b125-4ad7-b28e-efa3a8ab6068 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:56.683542 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11291a-b125-4ad7-b28e-efa3a8ab6068 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:56.684183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11291a-b125-4ad7-b28e-efa3a8ab6068 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:56.685659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11291a-b125-4ad7-b28e-efa3a8ab6068 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:56.711150 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 572/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:26:56 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:26:58.893695 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec78c86-9ddd-4081-a1b8-a2b8dcd4321f None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:58.894207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec78c86-9ddd-4081-a1b8-a2b8dcd4321f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:58.894207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec78c86-9ddd-4081-a1b8-a2b8dcd4321f None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:58.895496 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 602/1174] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:26:58.900783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9438a3-abc8-47dc-8de5-a23e776e121c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:58.901181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9438a3-abc8-47dc-8de5-a23e776e121c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:58.901366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9438a3-abc8-47dc-8de5-a23e776e121c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:58.919423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ac9438a3-abc8-47dc-8de5-a23e776e121c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:26:58.920596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ac9438a3-abc8-47dc-8de5-a23e776e121c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:26:58.986671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 573/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:26:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:26:59.257795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec78c86-9ddd-4081-a1b8-a2b8dcd4321f None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:59.338951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:59.365252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec78c86-9ddd-4081-a1b8-a2b8dcd4321f None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:59.398862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b55e5d-1f62-45b8-89bc-3a8bac75322e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:59.399487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b55e5d-1f62-45b8-89bc-3a8bac75322e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:59.399769 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b55e5d-1f62-45b8-89bc-3a8bac75322e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:59.400044 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b55e5d-1f62-45b8-89bc-3a8bac75322e None admin] PATH_INFO: `/v3/domains/5126f54e81364029ac1ae46a51ce6ee7` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:59.401176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b55e5d-1f62-45b8-89bc-3a8bac75322e None admin] RBAC: Authorizing `identity:get_domain(domain_id=5126f54e81364029ac1ae46a51ce6ee7)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:59.402589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b55e5d-1f62-45b8-89bc-3a8bac75322e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:59.404401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 603/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Sep 28 08:26:59 2022] GET /identity/v3/domains/5126f54e81364029ac1ae46a51ce6ee7 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:26:59.410124 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9438a3-abc8-47dc-8de5-a23e776e121c None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:26:59.439978 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:26:59.516602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9438a3-abc8-47dc-8de5-a23e776e121c None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:26:59.552273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf06bc-3a86-4d2a-b364-fd45f75903ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:26:59.552871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf06bc-3a86-4d2a-b364-fd45f75903ed None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:26:59.553190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf06bc-3a86-4d2a-b364-fd45f75903ed None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:26:59.553464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf06bc-3a86-4d2a-b364-fd45f75903ed None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:26:59.554123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf06bc-3a86-4d2a-b364-fd45f75903ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:26:59.555549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf06bc-3a86-4d2a-b364-fd45f75903ed None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:26:59.576761 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 574/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:26:59 2022] POST /identity/v3/users => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:27:01.772200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a1c17-f6ce-4dea-9843-92f4f4f8a170 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:01.772790 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a1c17-f6ce-4dea-9843-92f4f4f8a170 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:01.772790 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a1c17-f6ce-4dea-9843-92f4f4f8a170 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:01.774383 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 604/1178] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:27:01.780829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c016e-157d-4185-aa5c-b11c7daa1054 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:01.781275 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c016e-157d-4185-aa5c-b11c7daa1054 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:01.781383 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c016e-157d-4185-aa5c-b11c7daa1054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:01.799012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-d91c016e-157d-4185-aa5c-b11c7daa1054 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:27:01.800177 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d91c016e-157d-4185-aa5c-b11c7daa1054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:27:01.864452 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 575/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:27:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:27:02.136190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a1c17-f6ce-4dea-9843-92f4f4f8a170 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:02.238441 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a1c17-f6ce-4dea-9843-92f4f4f8a170 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:02.271674 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ded25-554f-48e6-bba3-7e8285bfecb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:02.272303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ded25-554f-48e6-bba3-7e8285bfecb9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:02.272581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ded25-554f-48e6-bba3-7e8285bfecb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:02.272857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ded25-554f-48e6-bba3-7e8285bfecb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:02.283614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ded25-554f-48e6-bba3-7e8285bfecb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-2a2ded25-554f-48e6-bba3-7e8285bfecb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:27:02.288409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:27:02.289514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:27:02.290555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 605/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:27:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:27:02.295896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c016e-157d-4185-aa5c-b11c7daa1054 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:02.313479 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:02.392513 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c016e-157d-4185-aa5c-b11c7daa1054 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:02.424953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a45486-4575-495b-af5d-a27f9125bb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:02.425576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a45486-4575-495b-af5d-a27f9125bb1b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:02.425850 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a45486-4575-495b-af5d-a27f9125bb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:02.426194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a45486-4575-495b-af5d-a27f9125bb1b None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:02.426766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a45486-4575-495b-af5d-a27f9125bb1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:27:02.427061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a45486-4575-495b-af5d-a27f9125bb1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:02.428417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a45486-4575-495b-af5d-a27f9125bb1b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:02.436637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 576/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:27:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:27:02.442868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ded25-554f-48e6-bba3-7e8285bfecb9 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:02.446284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:02.542809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ded25-554f-48e6-bba3-7e8285bfecb9 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:02.576917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c40c1-fcd5-4067-805b-e3a80e4c2514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:02.577574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c40c1-fcd5-4067-805b-e3a80e4c2514 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:02.577858 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c40c1-fcd5-4067-805b-e3a80e4c2514 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:02.578133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c40c1-fcd5-4067-805b-e3a80e4c2514 None admin] PATH_INFO: `/v3/users/9488bd01b2b64e94bb155a64633118c9` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:02.591576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c40c1-fcd5-4067-805b-e3a80e4c2514 None admin] RBAC: Authorizing `identity:get_user(user_id=9488bd01b2b64e94bb155a64633118c9)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:02.593046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c40c1-fcd5-4067-805b-e3a80e4c2514 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:02.594731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 606/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:27:02 2022] GET /identity/v3/users/9488bd01b2b64e94bb155a64633118c9 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:27:02.600564 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a45486-4575-495b-af5d-a27f9125bb1b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:02.682480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:02.702536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a45486-4575-495b-af5d-a27f9125bb1b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:02.735473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c786cf-ac08-4c70-9869-4b2999a95a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:02.736069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c786cf-ac08-4c70-9869-4b2999a95a6d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:02.736347 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c786cf-ac08-4c70-9869-4b2999a95a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:02.736617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c786cf-ac08-4c70-9869-4b2999a95a6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:02.803917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c786cf-ac08-4c70-9869-4b2999a95a6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:02.805394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c786cf-ac08-4c70-9869-4b2999a95a6d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:02.882403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:02.902400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 577/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:27:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:27:02.908631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c40c1-fcd5-4067-805b-e3a80e4c2514 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:02.965618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:03.005757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c40c1-fcd5-4067-805b-e3a80e4c2514 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:03.037993 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f436919-bef4-4602-89d7-670591631d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:03.038608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f436919-bef4-4602-89d7-670591631d73 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:03.038883 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f436919-bef4-4602-89d7-670591631d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:03.039149 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f436919-bef4-4602-89d7-670591631d73 None admin] PATH_INFO: `/v3/projects/2b70eca49f5c43b49ade66d0c37051dd` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:03.040232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f436919-bef4-4602-89d7-670591631d73 None admin] RBAC: Authorizing `identity:get_project(project_id=2b70eca49f5c43b49ade66d0c37051dd)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:03.041676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f436919-bef4-4602-89d7-670591631d73 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:03.043286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 607/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:27:02 2022] GET /identity/v3/projects/2b70eca49f5c43b49ade66d0c37051dd => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:27:03.049950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c786cf-ac08-4c70-9869-4b2999a95a6d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:03.145750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c786cf-ac08-4c70-9869-4b2999a95a6d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:03.179665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbe0e3-2ae6-4d7c-b0c7-05d6a56d8229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:03.180361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbe0e3-2ae6-4d7c-b0c7-05d6a56d8229 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:03.180642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbe0e3-2ae6-4d7c-b0c7-05d6a56d8229 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:03.180911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbe0e3-2ae6-4d7c-b0c7-05d6a56d8229 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:03.181646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbe0e3-2ae6-4d7c-b0c7-05d6a56d8229 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=2b70eca49f5c43b49ade66d0c37051dd, user.id=9488bd01b2b64e94bb155a64633118c9) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:27:03.181952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbe0e3-2ae6-4d7c-b0c7-05d6a56d8229 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:03.183357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fbe0e3-2ae6-4d7c-b0c7-05d6a56d8229 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:03.188563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 578/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:27:03 2022] GET /identity/v3/role_assignments?user.id=9488bd01b2b64e94bb155a64633118c9&scope.project.id=2b70eca49f5c43b49ade66d0c37051dd&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:27:05.342848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95827811-31fb-4829-8bbd-fb75a2a06599 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:05.343375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95827811-31fb-4829-8bbd-fb75a2a06599 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:05.343375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95827811-31fb-4829-8bbd-fb75a2a06599 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:05.344721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 608/1186] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:27:05.350299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d36f7d-3026-4410-ac67-092459bffbc7 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:05.350691 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d36f7d-3026-4410-ac67-092459bffbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:05.350837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d36f7d-3026-4410-ac67-092459bffbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:05.369291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-91d36f7d-3026-4410-ac67-092459bffbc7 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:27:05.370640 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-91d36f7d-3026-4410-ac67-092459bffbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:27:05.410087 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:05.437077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 579/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:27:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:27:05.705389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95827811-31fb-4829-8bbd-fb75a2a06599 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:05.806572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95827811-31fb-4829-8bbd-fb75a2a06599 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:05.832719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:05.840965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dad9c-a089-4668-b9fc-27f8799839a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:05.841658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dad9c-a089-4668-b9fc-27f8799839a4 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:05.841935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dad9c-a089-4668-b9fc-27f8799839a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:05.842202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dad9c-a089-4668-b9fc-27f8799839a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:05.852129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76dad9c-a089-4668-b9fc-27f8799839a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-d76dad9c-a089-4668-b9fc-27f8799839a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:27:05.856895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:27:05.858500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:27:05.859520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 609/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:27:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:27:05.864755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d36f7d-3026-4410-ac67-092459bffbc7 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:05.961188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d36f7d-3026-4410-ac67-092459bffbc7 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:05.994149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb214d6-5467-427e-83f7-a7eff8db25a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:05.994740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb214d6-5467-427e-83f7-a7eff8db25a6 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:05.995016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb214d6-5467-427e-83f7-a7eff8db25a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:05.995283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb214d6-5467-427e-83f7-a7eff8db25a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:05.995935 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb214d6-5467-427e-83f7-a7eff8db25a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:27:05.996235 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb214d6-5467-427e-83f7-a7eff8db25a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:05.997614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb214d6-5467-427e-83f7-a7eff8db25a6 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:06.005388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 580/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:27:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:27:06.011221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dad9c-a089-4668-b9fc-27f8799839a4 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:06.109444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dad9c-a089-4668-b9fc-27f8799839a4 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:06.143895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fefbb-a263-4451-a531-249d85f1cfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:06.144519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fefbb-a263-4451-a531-249d85f1cfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:06.144798 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fefbb-a263-4451-a531-249d85f1cfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:06.145072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fefbb-a263-4451-a531-249d85f1cfe2 None admin] PATH_INFO: `/v3/users/9488bd01b2b64e94bb155a64633118c9` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:06.146217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2fefbb-a263-4451-a531-249d85f1cfe2 None admin] RBAC: Authorizing `identity:get_user(user_id=9488bd01b2b64e94bb155a64633118c9)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:06.147641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2fefbb-a263-4451-a531-249d85f1cfe2 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:06.149223 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 610/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:27:06 2022] GET /identity/v3/users/9488bd01b2b64e94bb155a64633118c9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:27:06.156025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb214d6-5467-427e-83f7-a7eff8db25a6 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:06.190802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:06.253623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb214d6-5467-427e-83f7-a7eff8db25a6 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:06.286545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736d48b-c9c5-4e2f-9fb9-08c053d89805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:06.287176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736d48b-c9c5-4e2f-9fb9-08c053d89805 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:06.287453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736d48b-c9c5-4e2f-9fb9-08c053d89805 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:06.287723 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736d48b-c9c5-4e2f-9fb9-08c053d89805 None admin] PATH_INFO: `/v3/projects/2b70eca49f5c43b49ade66d0c37051dd` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:06.288816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736d48b-c9c5-4e2f-9fb9-08c053d89805 None admin] RBAC: Authorizing `identity:get_project(project_id=2b70eca49f5c43b49ade66d0c37051dd)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:06.290270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736d48b-c9c5-4e2f-9fb9-08c053d89805 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:06.291884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 581/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:27:06 2022] GET /identity/v3/projects/2b70eca49f5c43b49ade66d0c37051dd => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:27:06.298561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fefbb-a263-4451-a531-249d85f1cfe2 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:06.360155 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:06.403226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fefbb-a263-4451-a531-249d85f1cfe2 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:06.437450 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b0804-4e5e-4789-9461-3f8fdd805440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:06.438099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0804-4e5e-4789-9461-3f8fdd805440 None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:06.438378 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0804-4e5e-4789-9461-3f8fdd805440 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:06.438651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0804-4e5e-4789-9461-3f8fdd805440 None admin] PATH_INFO: `/v3/projects/2b70eca49f5c43b49ade66d0c37051dd/users/9488bd01b2b64e94bb155a64633118c9/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:06.440530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b0804-4e5e-4789-9461-3f8fdd805440 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b70eca49f5c43b49ade66d0c37051dd, user_id=9488bd01b2b64e94bb155a64633118c9, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:06.441978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b0804-4e5e-4789-9461-3f8fdd805440 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:06.454846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 611/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:27:06 2022] PUT /identity/v3/projects/2b70eca49f5c43b49ade66d0c37051dd/users/9488bd01b2b64e94bb155a64633118c9/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:27:08.628594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48773e3a-5cbd-4b2a-9ae5-b4df1896b8e8 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:08.629194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48773e3a-5cbd-4b2a-9ae5-b4df1896b8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:08.629194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48773e3a-5cbd-4b2a-9ae5-b4df1896b8e8 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:08.630516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 582/1193] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:27:08.637940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a3f30-b999-4cae-a890-f8570b746d73 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:08.638465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a3f30-b999-4cae-a890-f8570b746d73 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:08.638579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a3f30-b999-4cae-a890-f8570b746d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:08.658560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-6d2a3f30-b999-4cae-a890-f8570b746d73 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:27:08.659986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-6d2a3f30-b999-4cae-a890-f8570b746d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:27:08.735566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 612/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:27:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:27:09.007891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48773e3a-5cbd-4b2a-9ae5-b4df1896b8e8 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:09.110706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48773e3a-5cbd-4b2a-9ae5-b4df1896b8e8 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:09.113071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:09.148354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cad7c-3299-4e1e-ae01-dc299aa026fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:09.148984 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cad7c-3299-4e1e-ae01-dc299aa026fd None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:09.149303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cad7c-3299-4e1e-ae01-dc299aa026fd None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:09.149578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cad7c-3299-4e1e-ae01-dc299aa026fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:09.161311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cad7c-3299-4e1e-ae01-dc299aa026fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-ed2cad7c-3299-4e1e-ae01-dc299aa026fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:27:09.168080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:27:09.169267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:27:09.170420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 583/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:27:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:27:09.174997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a3f30-b999-4cae-a890-f8570b746d73 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:09.221066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:09.283307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a3f30-b999-4cae-a890-f8570b746d73 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:09.316590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823aa5df-b8cc-4892-a01a-12429cbaed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:09.317224 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823aa5df-b8cc-4892-a01a-12429cbaed25 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:09.317503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823aa5df-b8cc-4892-a01a-12429cbaed25 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:09.317775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823aa5df-b8cc-4892-a01a-12429cbaed25 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:09.318437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823aa5df-b8cc-4892-a01a-12429cbaed25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:27:09.318742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823aa5df-b8cc-4892-a01a-12429cbaed25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:09.320098 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823aa5df-b8cc-4892-a01a-12429cbaed25 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:09.328429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 613/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:27:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:27:09.334619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cad7c-3299-4e1e-ae01-dc299aa026fd None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:09.432666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cad7c-3299-4e1e-ae01-dc299aa026fd None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:09.466914 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a98f25-ad36-4008-b66b-6c6e5ca02172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:09.467546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a98f25-ad36-4008-b66b-6c6e5ca02172 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:09.467827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a98f25-ad36-4008-b66b-6c6e5ca02172 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:09.468102 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a98f25-ad36-4008-b66b-6c6e5ca02172 None admin] PATH_INFO: `/v3/users/9488bd01b2b64e94bb155a64633118c9` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:09.469239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a98f25-ad36-4008-b66b-6c6e5ca02172 None admin] RBAC: Authorizing `identity:get_user(user_id=9488bd01b2b64e94bb155a64633118c9)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:09.470667 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a98f25-ad36-4008-b66b-6c6e5ca02172 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:09.472248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 584/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 08:27:09 2022] GET /identity/v3/users/9488bd01b2b64e94bb155a64633118c9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:27:09.478170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823aa5df-b8cc-4892-a01a-12429cbaed25 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:09.579462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823aa5df-b8cc-4892-a01a-12429cbaed25 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:09.606616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:09.634103 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e3b64-52e8-4877-b6af-75545b2c3171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:09.634715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e3b64-52e8-4877-b6af-75545b2c3171 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:09.634999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e3b64-52e8-4877-b6af-75545b2c3171 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:09.635271 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e3b64-52e8-4877-b6af-75545b2c3171 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:09.701327 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e3b64-52e8-4877-b6af-75545b2c3171 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:09.702762 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e3b64-52e8-4877-b6af-75545b2c3171 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:09.798325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 614/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:27:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:27:09.804523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a98f25-ad36-4008-b66b-6c6e5ca02172 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:09.835599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:09.904540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a98f25-ad36-4008-b66b-6c6e5ca02172 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:09.944062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7aa9f4-af5c-4905-981e-43558bcb0939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:09.944679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7aa9f4-af5c-4905-981e-43558bcb0939 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:09.944960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7aa9f4-af5c-4905-981e-43558bcb0939 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:09.945272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7aa9f4-af5c-4905-981e-43558bcb0939 None admin] PATH_INFO: `/v3/projects/2b70eca49f5c43b49ade66d0c37051dd` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:09.946382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7aa9f4-af5c-4905-981e-43558bcb0939 None admin] RBAC: Authorizing `identity:get_project(project_id=2b70eca49f5c43b49ade66d0c37051dd)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:09.947932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7aa9f4-af5c-4905-981e-43558bcb0939 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:09.949600 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 585/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 08:27:09 2022] GET /identity/v3/projects/2b70eca49f5c43b49ade66d0c37051dd => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:27:09.956611 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e3b64-52e8-4877-b6af-75545b2c3171 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:27:09.967571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:10.054280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e3b64-52e8-4877-b6af-75545b2c3171 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:27:10.087699 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc8343-6507-45db-a2e1-6f7c2303e13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:27:10.088213 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8343-6507-45db-a2e1-6f7c2303e13d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:27:10.088497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8343-6507-45db-a2e1-6f7c2303e13d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:27:10.088769 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc8343-6507-45db-a2e1-6f7c2303e13d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:27:10.089515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc8343-6507-45db-a2e1-6f7c2303e13d None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=2b70eca49f5c43b49ade66d0c37051dd, user.id=9488bd01b2b64e94bb155a64633118c9) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:27:10.089824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc8343-6507-45db-a2e1-6f7c2303e13d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:27:10.091233 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc8343-6507-45db-a2e1-6f7c2303e13d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:27:10.096829 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 615/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:27:09 2022] GET /identity/v3/role_assignments?user.id=9488bd01b2b64e94bb155a64633118c9&scope.project.id=2b70eca49f5c43b49ade66d0c37051dd&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:27:19.938906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=50 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:27:20.094612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=64 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:07.374560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e7c0f-2d4b-44dc-8f13-b4a01e7ca564 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:07.375175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e7c0f-2d4b-44dc-8f13-b4a01e7ca564 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:07.375175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e7c0f-2d4b-44dc-8f13-b4a01e7ca564 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:07.376434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 586/1201] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:28:07.382110 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a5c97-5428-4b61-b1ae-083cab527f11 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:07.382625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a5c97-5428-4b61-b1ae-083cab527f11 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:07.382726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a5c97-5428-4b61-b1ae-083cab527f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:07.405139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-0d3a5c97-5428-4b61-b1ae-083cab527f11 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:28:07.405374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-0d3a5c97-5428-4b61-b1ae-083cab527f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:28:07.487380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 616/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:28:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:28:07.764295 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e7c0f-2d4b-44dc-8f13-b4a01e7ca564 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:07.905106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e7c0f-2d4b-44dc-8f13-b4a01e7ca564 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:07.947420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fac65-9c4b-4893-abc6-ac37ac88a51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:07.948108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fac65-9c4b-4893-abc6-ac37ac88a51d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:07.948380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fac65-9c4b-4893-abc6-ac37ac88a51d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:07.948649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fac65-9c4b-4893-abc6-ac37ac88a51d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:07.961755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fac65-9c4b-4893-abc6-ac37ac88a51d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:07.966954 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fac65-9c4b-4893-abc6-ac37ac88a51d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-ac3fac65-9c4b-4893-abc6-ac37ac88a51d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:07.982313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:07.985548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:07.986576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:07.986576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:07.986576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:07.986576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 587/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:07.992682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a5c97-5428-4b61-b1ae-083cab527f11 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:08.136503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a5c97-5428-4b61-b1ae-083cab527f11 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:08.183228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:08.192521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299af2c1-d349-48a0-8f5f-81ff90fc3bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:08.193759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299af2c1-d349-48a0-8f5f-81ff90fc3bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:08.194344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299af2c1-d349-48a0-8f5f-81ff90fc3bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:08.194908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299af2c1-d349-48a0-8f5f-81ff90fc3bd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:08.196627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299af2c1-d349-48a0-8f5f-81ff90fc3bd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:08.197173 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299af2c1-d349-48a0-8f5f-81ff90fc3bd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:08.198930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299af2c1-d349-48a0-8f5f-81ff90fc3bd6 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:08.228675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 617/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:28:08.235317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fac65-9c4b-4893-abc6-ac37ac88a51d None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:08.328893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:08.399918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fac65-9c4b-4893-abc6-ac37ac88a51d None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:08.441870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c2680-64a9-48bc-b382-d7cbd53891da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:08.442621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c2680-64a9-48bc-b382-d7cbd53891da None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:08.442967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c2680-64a9-48bc-b382-d7cbd53891da None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:08.443253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c2680-64a9-48bc-b382-d7cbd53891da None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:08.443931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c2680-64a9-48bc-b382-d7cbd53891da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:08.445675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c2680-64a9-48bc-b382-d7cbd53891da None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:08.472364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 588/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 08:28:08 2022] POST /identity/v3/users => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:28:10.966909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de40aa-acb6-4239-a343-537e0794e4fc None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:10.967520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de40aa-acb6-4239-a343-537e0794e4fc None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:10.967520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de40aa-acb6-4239-a343-537e0794e4fc None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:10.968905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 618/1206] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:28:10.974717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb981e0-e5bf-4f09-9a2d-53b57459e8a8 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:10.975270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb981e0-e5bf-4f09-9a2d-53b57459e8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:10.975270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb981e0-e5bf-4f09-9a2d-53b57459e8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:10.994804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-3bb981e0-e5bf-4f09-9a2d-53b57459e8a8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:28:10.996015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3bb981e0-e5bf-4f09-9a2d-53b57459e8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:28:11.072231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 589/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:28:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:28:11.348475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de40aa-acb6-4239-a343-537e0794e4fc None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:11.480742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de40aa-acb6-4239-a343-537e0794e4fc None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:11.527565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd7780-766b-47bc-9b5e-4a92fdfefc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:11.528779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7780-766b-47bc-9b5e-4a92fdfefc2f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:11.529352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7780-766b-47bc-9b5e-4a92fdfefc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:11.530028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd7780-766b-47bc-9b5e-4a92fdfefc2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:11.557010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd7780-766b-47bc-9b5e-4a92fdfefc2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-f9dd7780-766b-47bc-9b5e-4a92fdfefc2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:28:11.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:11.563937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:11.564851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 619/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:28:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:11.572050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb981e0-e5bf-4f09-9a2d-53b57459e8a8 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:11.637288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:11.718407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb981e0-e5bf-4f09-9a2d-53b57459e8a8 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:11.768026 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672fd26-fce7-4797-bb3a-1a20b6d02468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:11.768702 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672fd26-fce7-4797-bb3a-1a20b6d02468 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:11.768974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672fd26-fce7-4797-bb3a-1a20b6d02468 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:11.769279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672fd26-fce7-4797-bb3a-1a20b6d02468 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:11.769959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672fd26-fce7-4797-bb3a-1a20b6d02468 None admin] RBAC: Adding query filter params (name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:11.770265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672fd26-fce7-4797-bb3a-1a20b6d02468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:11.771645 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672fd26-fce7-4797-bb3a-1a20b6d02468 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:11.783268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 590/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:28:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:11.789698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd7780-766b-47bc-9b5e-4a92fdfefc2f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:11.855622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:11.923333 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd7780-766b-47bc-9b5e-4a92fdfefc2f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:11.966263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29251224-e561-49b0-bc07-09096039f764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:11.966952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29251224-e561-49b0-bc07-09096039f764 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:11.967638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29251224-e561-49b0-bc07-09096039f764 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:11.967638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29251224-e561-49b0-bc07-09096039f764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:11.980140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29251224-e561-49b0-bc07-09096039f764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:11.981792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29251224-e561-49b0-bc07-09096039f764 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-29251224-e561-49b0-bc07-09096039f764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:11.999849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:12.001159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:12.002260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:12.002260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:12.002260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:12.005991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 620/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:12.007012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672fd26-fce7-4797-bb3a-1a20b6d02468 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:12.159681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:12.163855 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672fd26-fce7-4797-bb3a-1a20b6d02468 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:12.213655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81525b6-4ea1-4bb5-a98c-24ec9b9e319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:12.214280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81525b6-4ea1-4bb5-a98c-24ec9b9e319f None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:12.214561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81525b6-4ea1-4bb5-a98c-24ec9b9e319f None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:12.214840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81525b6-4ea1-4bb5-a98c-24ec9b9e319f None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:12.215519 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81525b6-4ea1-4bb5-a98c-24ec9b9e319f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:12.215824 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81525b6-4ea1-4bb5-a98c-24ec9b9e319f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:12.217257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81525b6-4ea1-4bb5-a98c-24ec9b9e319f None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:12.231043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 591/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:28:12.238473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29251224-e561-49b0-bc07-09096039f764 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:12.362641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29251224-e561-49b0-bc07-09096039f764 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:12.364326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:12.407906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426adac8-9169-43a5-ae4f-a8b6cd6a58fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:12.409053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426adac8-9169-43a5-ae4f-a8b6cd6a58fe None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:12.409053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426adac8-9169-43a5-ae4f-a8b6cd6a58fe None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:12.409318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426adac8-9169-43a5-ae4f-a8b6cd6a58fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:12.419814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426adac8-9169-43a5-ae4f-a8b6cd6a58fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:12.421506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426adac8-9169-43a5-ae4f-a8b6cd6a58fe None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-426adac8-9169-43a5-ae4f-a8b6cd6a58fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:28:12.435357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:12.436841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:12.438075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:12.438075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:12.438075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:28:12.438075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:28:12.438075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:28:12.438075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:28:12.438075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:28:12.438075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:28:12.438075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 08:28:12.438075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:12.439114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 621/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:12.444974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81525b6-4ea1-4bb5-a98c-24ec9b9e319f None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:12.578758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81525b6-4ea1-4bb5-a98c-24ec9b9e319f None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:12.626200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da946504-c620-493b-bd8a-1b62c8ca243a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:12.626841 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da946504-c620-493b-bd8a-1b62c8ca243a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:12.627124 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da946504-c620-493b-bd8a-1b62c8ca243a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:12.627397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da946504-c620-493b-bd8a-1b62c8ca243a None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:12.628140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da946504-c620-493b-bd8a-1b62c8ca243a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:12.628444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da946504-c620-493b-bd8a-1b62c8ca243a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:12.629928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da946504-c620-493b-bd8a-1b62c8ca243a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:12.650091 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 592/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 08:28:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:12.657312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426adac8-9169-43a5-ae4f-a8b6cd6a58fe None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:12.850649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426adac8-9169-43a5-ae4f-a8b6cd6a58fe None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:12.924742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30dc88-b441-4f0f-8df9-b5c364477070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:12.925935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30dc88-b441-4f0f-8df9-b5c364477070 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:12.926435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30dc88-b441-4f0f-8df9-b5c364477070 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:12.926923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30dc88-b441-4f0f-8df9-b5c364477070 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:12.968146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:13.023951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30dc88-b441-4f0f-8df9-b5c364477070 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:13.026137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30dc88-b441-4f0f-8df9-b5c364477070 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:13.150247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 622/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:28:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 08:28:13.157806 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da946504-c620-493b-bd8a-1b62c8ca243a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:13.179529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:13.285867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da946504-c620-493b-bd8a-1b62c8ca243a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:13.329948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2ec93-adb8-4b62-bbb3-ee9ef2bdb7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:13.330454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2ec93-adb8-4b62-bbb3-ee9ef2bdb7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:13.330735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2ec93-adb8-4b62-bbb3-ee9ef2bdb7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:13.331039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2ec93-adb8-4b62-bbb3-ee9ef2bdb7b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:13.345639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2ec93-adb8-4b62-bbb3-ee9ef2bdb7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:13.347088 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2ec93-adb8-4b62-bbb3-ee9ef2bdb7b7 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-c2f2ec93-adb8-4b62-bbb3-ee9ef2bdb7b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:13.371677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:13.372756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:13.373662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:13.373662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:13.373662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:13.373662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 593/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:28:13.375693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30dc88-b441-4f0f-8df9-b5c364477070 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:13.454136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:13.507742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30dc88-b441-4f0f-8df9-b5c364477070 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:13.550954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b301c10-fadf-4a5f-9f61-7fcbb523f667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:13.551580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b301c10-fadf-4a5f-9f61-7fcbb523f667 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:13.551860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b301c10-fadf-4a5f-9f61-7fcbb523f667 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:13.552131 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b301c10-fadf-4a5f-9f61-7fcbb523f667 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:13.552853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b301c10-fadf-4a5f-9f61-7fcbb523f667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:13.553198 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b301c10-fadf-4a5f-9f61-7fcbb523f667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:13.554600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b301c10-fadf-4a5f-9f61-7fcbb523f667 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:13.567901 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 623/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:13.576246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2ec93-adb8-4b62-bbb3-ee9ef2bdb7b7 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:13.661589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:13.712654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2ec93-adb8-4b62-bbb3-ee9ef2bdb7b7 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:13.760612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2e69b-39ec-4168-991c-d49883fca3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:13.761399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2e69b-39ec-4168-991c-d49883fca3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:13.761657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2e69b-39ec-4168-991c-d49883fca3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:13.761927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2e69b-39ec-4168-991c-d49883fca3d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:13.770551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2e69b-39ec-4168-991c-d49883fca3d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:13.772692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2e69b-39ec-4168-991c-d49883fca3d4 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-fae2e69b-39ec-4168-991c-d49883fca3d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:28:13.787104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:13.788408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:13.789344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:28:13.789344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:28:13.789344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:28:13.789344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:28:13.789344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:28:13.789344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:28:13.789344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:28:13.789344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:13.789344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 594/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:28:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:13.794793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b301c10-fadf-4a5f-9f61-7fcbb523f667 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:13.938865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b301c10-fadf-4a5f-9f61-7fcbb523f667 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:13.963784 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:13.976063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f95c9-159a-4bcf-a503-fd11ad9a5715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:13.976679 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f95c9-159a-4bcf-a503-fd11ad9a5715 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:13.976936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f95c9-159a-4bcf-a503-fd11ad9a5715 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:13.977241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f95c9-159a-4bcf-a503-fd11ad9a5715 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:13.977933 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f95c9-159a-4bcf-a503-fd11ad9a5715 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:13.978243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f95c9-159a-4bcf-a503-fd11ad9a5715 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:13.979648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f95c9-159a-4bcf-a503-fd11ad9a5715 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:13.999419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 624/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:28:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:14.003058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2e69b-39ec-4168-991c-d49883fca3d4 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:14.128264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2e69b-39ec-4168-991c-d49883fca3d4 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:14.163198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:14.170200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c06129-6552-435b-b0c6-c7721c6792db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:14.170844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c06129-6552-435b-b0c6-c7721c6792db None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:14.171117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c06129-6552-435b-b0c6-c7721c6792db None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:14.171390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c06129-6552-435b-b0c6-c7721c6792db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:14.172132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c06129-6552-435b-b0c6-c7721c6792db None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=6b40ee263df74e58bf3f958c01eb742f) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:14.172429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c06129-6552-435b-b0c6-c7721c6792db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:14.173911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c06129-6552-435b-b0c6-c7721c6792db None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:14.180721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 595/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:28:14 2022] GET /identity/v3/role_assignments?user.id=6b40ee263df74e58bf3f958c01eb742f&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:16.465064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947c7a5-2c72-474c-bcaa-8a39aa84cfed None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:16.465697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947c7a5-2c72-474c-bcaa-8a39aa84cfed None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:16.465697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947c7a5-2c72-474c-bcaa-8a39aa84cfed None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:16.467050 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 625/1220] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:28:16.473641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b7e6c-8cb9-48df-91f5-a3bb064da10d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:16.474181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b7e6c-8cb9-48df-91f5-a3bb064da10d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:16.474358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b7e6c-8cb9-48df-91f5-a3bb064da10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:16.495465 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-be9b7e6c-8cb9-48df-91f5-a3bb064da10d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:28:16.496857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-be9b7e6c-8cb9-48df-91f5-a3bb064da10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:28:16.577959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 596/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:28:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:28:16.880747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947c7a5-2c72-474c-bcaa-8a39aa84cfed None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:17.003983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947c7a5-2c72-474c-bcaa-8a39aa84cfed None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:17.043582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e298d-d58b-477b-aabc-ae54dbdc69a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:17.044225 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e298d-d58b-477b-aabc-ae54dbdc69a4 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:17.044506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e298d-d58b-477b-aabc-ae54dbdc69a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:17.044779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e298d-d58b-477b-aabc-ae54dbdc69a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:17.056396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e298d-d58b-477b-aabc-ae54dbdc69a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-ed2e298d-d58b-477b-aabc-ae54dbdc69a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:28:17.061501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:17.063688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:17.065708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 626/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:28:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:17.071840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b7e6c-8cb9-48df-91f5-a3bb064da10d None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:17.210123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b7e6c-8cb9-48df-91f5-a3bb064da10d None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:17.236893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:17.252468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cbac3-6aa0-4972-9632-f268c7207338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:17.253584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cbac3-6aa0-4972-9632-f268c7207338 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:17.253948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cbac3-6aa0-4972-9632-f268c7207338 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:17.254243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cbac3-6aa0-4972-9632-f268c7207338 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:17.255096 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cbac3-6aa0-4972-9632-f268c7207338 None admin] RBAC: Adding query filter params (name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:17.255436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cbac3-6aa0-4972-9632-f268c7207338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:17.256998 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46cbac3-6aa0-4972-9632-f268c7207338 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:17.266931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 597/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:28:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:17.273113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e298d-d58b-477b-aabc-ae54dbdc69a4 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:17.332623 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:17.405392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e298d-d58b-477b-aabc-ae54dbdc69a4 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:17.450435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b4fef-ec5f-4fab-98a3-2de10858d7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:17.450435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b4fef-ec5f-4fab-98a3-2de10858d7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:17.451007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b4fef-ec5f-4fab-98a3-2de10858d7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:17.451007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b4fef-ec5f-4fab-98a3-2de10858d7f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:17.462815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b4fef-ec5f-4fab-98a3-2de10858d7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:17.464193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b4fef-ec5f-4fab-98a3-2de10858d7f9 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-3e9b4fef-ec5f-4fab-98a3-2de10858d7f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:17.480940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:17.483544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:17.484448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:17.484448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:17.484448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:17.485536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 627/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:28:17.488654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cbac3-6aa0-4972-9632-f268c7207338 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:17.631878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cbac3-6aa0-4972-9632-f268c7207338 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:17.674965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbad8d3-99d9-4120-9a7f-99b7996ac53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:17.675594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbad8d3-99d9-4120-9a7f-99b7996ac53b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:17.675882 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbad8d3-99d9-4120-9a7f-99b7996ac53b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:17.676153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbad8d3-99d9-4120-9a7f-99b7996ac53b None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:17.676888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbad8d3-99d9-4120-9a7f-99b7996ac53b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:17.677244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbad8d3-99d9-4120-9a7f-99b7996ac53b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:17.678775 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbad8d3-99d9-4120-9a7f-99b7996ac53b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:17.692751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 598/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:17.700520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b4fef-ec5f-4fab-98a3-2de10858d7f9 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:17.823139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:17.831237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b4fef-ec5f-4fab-98a3-2de10858d7f9 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:17.874937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5216c-89bb-4b6d-8b77-5d5f32a58a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:17.875744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5216c-89bb-4b6d-8b77-5d5f32a58a18 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:17.876068 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5216c-89bb-4b6d-8b77-5d5f32a58a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:17.876395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5216c-89bb-4b6d-8b77-5d5f32a58a18 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:17.885494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5216c-89bb-4b6d-8b77-5d5f32a58a18 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:17.887670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5216c-89bb-4b6d-8b77-5d5f32a58a18 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-caa5216c-89bb-4b6d-8b77-5d5f32a58a18 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:28:17.901909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:17.904533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:17.905935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:17.905935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:17.905935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:28:17.905935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:28:17.905935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:28:17.905935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:28:17.905935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:28:17.905935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:28:17.905935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 08:28:17.905935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:17.905935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 628/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:28:17.911451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbad8d3-99d9-4120-9a7f-99b7996ac53b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:17.983259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:18.052270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbad8d3-99d9-4120-9a7f-99b7996ac53b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:18.095478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc9bbe-dae4-4159-8eb5-a0cca14b9fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:18.098035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc9bbe-dae4-4159-8eb5-a0cca14b9fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:18.098035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc9bbe-dae4-4159-8eb5-a0cca14b9fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:18.098035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc9bbe-dae4-4159-8eb5-a0cca14b9fd2 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:18.098618 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc9bbe-dae4-4159-8eb5-a0cca14b9fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:18.099018 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc9bbe-dae4-4159-8eb5-a0cca14b9fd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:18.100570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc9bbe-dae4-4159-8eb5-a0cca14b9fd2 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:18.134608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 599/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 08:28:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:18.144859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5216c-89bb-4b6d-8b77-5d5f32a58a18 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:18.287022 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5216c-89bb-4b6d-8b77-5d5f32a58a18 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:18.327754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c877a41-3ab6-45ad-8cd1-7a138baee4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:18.328606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c877a41-3ab6-45ad-8cd1-7a138baee4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:18.328986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c877a41-3ab6-45ad-8cd1-7a138baee4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:18.329434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c877a41-3ab6-45ad-8cd1-7a138baee4e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:18.344333 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c877a41-3ab6-45ad-8cd1-7a138baee4e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:18.345926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c877a41-3ab6-45ad-8cd1-7a138baee4e7 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-2c877a41-3ab6-45ad-8cd1-7a138baee4e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:18.365183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:18.366362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:18.367541 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:18.367541 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:18.367541 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:18.369190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 629/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:18.379735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc9bbe-dae4-4159-8eb5-a0cca14b9fd2 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:18.519513 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc9bbe-dae4-4159-8eb5-a0cca14b9fd2 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:18.525679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:18.569305 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a494b2-daed-4534-b01f-c4f848222997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:18.570063 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a494b2-daed-4534-b01f-c4f848222997 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:18.570524 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a494b2-daed-4534-b01f-c4f848222997 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:18.570900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a494b2-daed-4534-b01f-c4f848222997 None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:18.571705 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a494b2-daed-4534-b01f-c4f848222997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:18.572138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a494b2-daed-4534-b01f-c4f848222997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:18.573856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a494b2-daed-4534-b01f-c4f848222997 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:18.587768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 600/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:28:18.594326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c877a41-3ab6-45ad-8cd1-7a138baee4e7 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:18.604440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:18.730434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c877a41-3ab6-45ad-8cd1-7a138baee4e7 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:18.769883 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2875980-7904-452d-959a-3a6e21d9956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:18.770644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2875980-7904-452d-959a-3a6e21d9956c None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:18.770925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2875980-7904-452d-959a-3a6e21d9956c None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:18.771195 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2875980-7904-452d-959a-3a6e21d9956c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:18.781722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2875980-7904-452d-959a-3a6e21d9956c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:18.784063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2875980-7904-452d-959a-3a6e21d9956c None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-c2875980-7904-452d-959a-3a6e21d9956c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:28:18.797201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:18.798337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:18.799394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:28:18.799394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:28:18.799394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:28:18.799394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:28:18.799394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:28:18.799394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:28:18.799394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:28:18.799394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:18.799394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 630/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:28:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:18.806016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a494b2-daed-4534-b01f-c4f848222997 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:18.931930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a494b2-daed-4534-b01f-c4f848222997 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:18.972793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd30f95-41cd-4844-9d7d-b03524d188c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:18.973781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd30f95-41cd-4844-9d7d-b03524d188c5 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:18.974176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd30f95-41cd-4844-9d7d-b03524d188c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:18.974554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd30f95-41cd-4844-9d7d-b03524d188c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:18.975596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd30f95-41cd-4844-9d7d-b03524d188c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:18.976032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd30f95-41cd-4844-9d7d-b03524d188c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:18.978571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd30f95-41cd-4844-9d7d-b03524d188c5 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:19.000011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 601/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:28:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:19.009917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2875980-7904-452d-959a-3a6e21d9956c None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:19.099616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:19.152028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2875980-7904-452d-959a-3a6e21d9956c None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:19.203344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdde40e-bedd-4d6e-bc96-3f1b5475f6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:19.204898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdde40e-bedd-4d6e-bc96-3f1b5475f6ff None admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:19.205706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdde40e-bedd-4d6e-bc96-3f1b5475f6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:19.206288 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdde40e-bedd-4d6e-bc96-3f1b5475f6ff None admin] PATH_INFO: `/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/6b40ee263df74e58bf3f958c01eb742f/roles/5354696222554ab2a19a3a839682a597` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:19.225515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdde40e-bedd-4d6e-bc96-3f1b5475f6ff None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca4d4801f2f40419f6881984fbbfbcd, user_id=6b40ee263df74e58bf3f958c01eb742f, role_id=5354696222554ab2a19a3a839682a597)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:19.226978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdde40e-bedd-4d6e-bc96-3f1b5475f6ff None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:19.251429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 631/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:28:19 2022] PUT /identity/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/6b40ee263df74e58bf3f958c01eb742f/roles/5354696222554ab2a19a3a839682a597 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:28:21.747548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dcdd2-aee2-4347-b68d-155098cfe888 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:21.748497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dcdd2-aee2-4347-b68d-155098cfe888 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:21.748497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dcdd2-aee2-4347-b68d-155098cfe888 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:21.749624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 602/1233] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:28:21.756960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e454c6-551f-4f51-975d-33d5a6d3f0ee None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:21.758157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e454c6-551f-4f51-975d-33d5a6d3f0ee None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:21.758438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e454c6-551f-4f51-975d-33d5a6d3f0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:21.782868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-95e454c6-551f-4f51-975d-33d5a6d3f0ee None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:28:21.784325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-95e454c6-551f-4f51-975d-33d5a6d3f0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:28:21.874417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 632/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:28:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:28:22.188964 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674dcdd2-aee2-4347-b68d-155098cfe888 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:22.240914 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:22.360390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674dcdd2-aee2-4347-b68d-155098cfe888 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:22.402642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c19890-6f39-47d3-8a6b-d34479214b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:22.403317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c19890-6f39-47d3-8a6b-d34479214b86 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:22.403599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c19890-6f39-47d3-8a6b-d34479214b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:22.403874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c19890-6f39-47d3-8a6b-d34479214b86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:22.418738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c19890-6f39-47d3-8a6b-d34479214b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-38c19890-6f39-47d3-8a6b-d34479214b86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:28:22.424127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:22.425676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:22.426887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 603/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 08:28:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:22.432808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e454c6-551f-4f51-975d-33d5a6d3f0ee None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:22.492047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:22.575558 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e454c6-551f-4f51-975d-33d5a6d3f0ee None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:22.624328 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41c81-011e-47cd-8a59-285c0545507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:22.624980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41c81-011e-47cd-8a59-285c0545507f None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:22.625363 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41c81-011e-47cd-8a59-285c0545507f None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:22.625673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d41c81-011e-47cd-8a59-285c0545507f None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:22.626627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d41c81-011e-47cd-8a59-285c0545507f None admin] RBAC: Adding query filter params (name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:22.626952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d41c81-011e-47cd-8a59-285c0545507f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:22.628672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d41c81-011e-47cd-8a59-285c0545507f None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:22.639133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 633/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 08:28:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:22.649599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c19890-6f39-47d3-8a6b-d34479214b86 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:22.763041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:22.784640 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c19890-6f39-47d3-8a6b-d34479214b86 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:22.828152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bce5a-3022-48dd-9f9a-cb7acd9ed77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:22.828968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bce5a-3022-48dd-9f9a-cb7acd9ed77c None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:22.829358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bce5a-3022-48dd-9f9a-cb7acd9ed77c None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:22.829674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bce5a-3022-48dd-9f9a-cb7acd9ed77c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:22.844378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bce5a-3022-48dd-9f9a-cb7acd9ed77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:22.845957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bce5a-3022-48dd-9f9a-cb7acd9ed77c None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-e80bce5a-3022-48dd-9f9a-cb7acd9ed77c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:22.864808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:22.866029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:22.867177 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:22.867177 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:22.867177 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:22.867177 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 604/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:22.872487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41c81-011e-47cd-8a59-285c0545507f None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:23.028987 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d41c81-011e-47cd-8a59-285c0545507f None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:23.041770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:23.069906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc44ef-709f-4dac-a3ee-8fa10b2624ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:23.070519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc44ef-709f-4dac-a3ee-8fa10b2624ce None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:23.070800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc44ef-709f-4dac-a3ee-8fa10b2624ce None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:23.071073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc44ef-709f-4dac-a3ee-8fa10b2624ce None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:23.071710 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc44ef-709f-4dac-a3ee-8fa10b2624ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:23.072014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc44ef-709f-4dac-a3ee-8fa10b2624ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:23.073416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc44ef-709f-4dac-a3ee-8fa10b2624ce None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:23.086905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 634/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:23.094441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bce5a-3022-48dd-9f9a-cb7acd9ed77c None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:23.240817 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bce5a-3022-48dd-9f9a-cb7acd9ed77c None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:23.288034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc8451-6413-4339-b3a7-acb4178ad716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:23.288601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc8451-6413-4339-b3a7-acb4178ad716 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:23.288880 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc8451-6413-4339-b3a7-acb4178ad716 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:23.289187 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc8451-6413-4339-b3a7-acb4178ad716 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:23.292729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:23.300294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc8451-6413-4339-b3a7-acb4178ad716 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:23.302147 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc8451-6413-4339-b3a7-acb4178ad716 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-b4fc8451-6413-4339-b3a7-acb4178ad716 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:28:23.316893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:23.318014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:23.319007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:23.319007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:23.319007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:28:23.319007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:28:23.319007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:28:23.319007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:28:23.319007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:28:23.319007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:28:23.319007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 08:28:23.319007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:23.319007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 605/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:28:23.325629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc44ef-709f-4dac-a3ee-8fa10b2624ce None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:23.450845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc44ef-709f-4dac-a3ee-8fa10b2624ce None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:23.498470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c221f-f8a9-48c7-8916-de787be5a3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:23.499381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c221f-f8a9-48c7-8916-de787be5a3bd None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:23.499723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c221f-f8a9-48c7-8916-de787be5a3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:23.500065 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c221f-f8a9-48c7-8916-de787be5a3bd None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:23.500953 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c221f-f8a9-48c7-8916-de787be5a3bd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:23.501382 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c221f-f8a9-48c7-8916-de787be5a3bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:23.502964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c221f-f8a9-48c7-8916-de787be5a3bd None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:23.530178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 635/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 08:28:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:23.540531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc8451-6413-4339-b3a7-acb4178ad716 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:23.683458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc8451-6413-4339-b3a7-acb4178ad716 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:23.725147 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960ae48-04d1-442d-b74a-cbf5fc8d39d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:23.725833 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960ae48-04d1-442d-b74a-cbf5fc8d39d4 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:23.726136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960ae48-04d1-442d-b74a-cbf5fc8d39d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:23.726439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960ae48-04d1-442d-b74a-cbf5fc8d39d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:23.800842 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:23.810306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960ae48-04d1-442d-b74a-cbf5fc8d39d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:23.811833 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960ae48-04d1-442d-b74a-cbf5fc8d39d4 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:23.931074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 606/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:28:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:28:23.937719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c221f-f8a9-48c7-8916-de787be5a3bd None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:23.968011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:24.078540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c221f-f8a9-48c7-8916-de787be5a3bd None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:24.125007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5711b8-ee76-4e85-84c7-79bd15b41a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:24.125786 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5711b8-ee76-4e85-84c7-79bd15b41a97 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:24.126092 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5711b8-ee76-4e85-84c7-79bd15b41a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:24.126388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5711b8-ee76-4e85-84c7-79bd15b41a97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:24.141608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5711b8-ee76-4e85-84c7-79bd15b41a97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:24.143262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5711b8-ee76-4e85-84c7-79bd15b41a97 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-4d5711b8-ee76-4e85-84c7-79bd15b41a97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:24.169205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:24.171812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:24.173066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:24.173066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:24.173066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:24.173066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 636/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:28:24.177288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960ae48-04d1-442d-b74a-cbf5fc8d39d4 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:24.309464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:24.316601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960ae48-04d1-442d-b74a-cbf5fc8d39d4 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:24.356361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7e9d9-e30a-4762-baac-5a05edf3977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:24.356910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7e9d9-e30a-4762-baac-5a05edf3977b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:24.357233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7e9d9-e30a-4762-baac-5a05edf3977b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:24.357489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7e9d9-e30a-4762-baac-5a05edf3977b None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:24.358157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7e9d9-e30a-4762-baac-5a05edf3977b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:24.358487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7e9d9-e30a-4762-baac-5a05edf3977b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:24.359825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7e9d9-e30a-4762-baac-5a05edf3977b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:24.373262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 607/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:28:24.379662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5711b8-ee76-4e85-84c7-79bd15b41a97 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:24.508685 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:24.536976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5711b8-ee76-4e85-84c7-79bd15b41a97 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:24.577748 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328066a9-585a-4283-8673-1e53b82e16ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:24.578415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328066a9-585a-4283-8673-1e53b82e16ad None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:24.578677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328066a9-585a-4283-8673-1e53b82e16ad None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:24.578949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328066a9-585a-4283-8673-1e53b82e16ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:24.586652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328066a9-585a-4283-8673-1e53b82e16ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:24.588130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328066a9-585a-4283-8673-1e53b82e16ad None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-328066a9-585a-4283-8673-1e53b82e16ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:28:24.605791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:24.607961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:24.609725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:28:24.609725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:28:24.609725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:28:24.609725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:28:24.609725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:28:24.609725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:28:24.609725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:28:24.609725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:24.609725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 637/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:28:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:28:24.616555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7e9d9-e30a-4762-baac-5a05edf3977b None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:24.748420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7e9d9-e30a-4762-baac-5a05edf3977b None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:24.801497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fd83a-e3b3-4f5f-817a-63fd68d7f18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:24.802284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fd83a-e3b3-4f5f-817a-63fd68d7f18b None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:24.802619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fd83a-e3b3-4f5f-817a-63fd68d7f18b None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:24.803024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fd83a-e3b3-4f5f-817a-63fd68d7f18b None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:24.803887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fd83a-e3b3-4f5f-817a-63fd68d7f18b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:24.804248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fd83a-e3b3-4f5f-817a-63fd68d7f18b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:24.806054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fd83a-e3b3-4f5f-817a-63fd68d7f18b None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:24.821985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 608/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:28:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:28:24.831311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328066a9-585a-4283-8673-1e53b82e16ad None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:24.960031 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328066a9-585a-4283-8673-1e53b82e16ad None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:25.002371 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283c971-38fa-4dbe-b86e-d7ea83271673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:25.003189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283c971-38fa-4dbe-b86e-d7ea83271673 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:25.003528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283c971-38fa-4dbe-b86e-d7ea83271673 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:25.004006 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283c971-38fa-4dbe-b86e-d7ea83271673 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:25.004957 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283c971-38fa-4dbe-b86e-d7ea83271673 None admin] RBAC: Adding query filter params (role.id=5354696222554ab2a19a3a839682a597, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=6b40ee263df74e58bf3f958c01eb742f) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:25.005341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283c971-38fa-4dbe-b86e-d7ea83271673 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:25.006969 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283c971-38fa-4dbe-b86e-d7ea83271673 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:25.029133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 638/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:28:24 2022] GET /identity/v3/role_assignments?user.id=6b40ee263df74e58bf3f958c01eb742f&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=5354696222554ab2a19a3a839682a597 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:27.508221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd30803-fc27-4e9d-8aba-10107893126a None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:27.508805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd30803-fc27-4e9d-8aba-10107893126a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:27.508805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd30803-fc27-4e9d-8aba-10107893126a None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:27.510601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 609/1247] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:28:27.519608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511bd63-b074-40d6-8515-d8d09aa0d2eb None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:27.521137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511bd63-b074-40d6-8515-d8d09aa0d2eb None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:27.521137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511bd63-b074-40d6-8515-d8d09aa0d2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:27.530727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:27.551622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b511bd63-b074-40d6-8515-d8d09aa0d2eb None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:28:27.553381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b511bd63-b074-40d6-8515-d8d09aa0d2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:28:27.657214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 639/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:28:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:28:27.957040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd30803-fc27-4e9d-8aba-10107893126a None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:27.982919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:28.104505 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd30803-fc27-4e9d-8aba-10107893126a None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:28.155839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76d525-edb7-445f-9954-f0c0dde6fb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:28.156630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76d525-edb7-445f-9954-f0c0dde6fb07 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:28.156948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76d525-edb7-445f-9954-f0c0dde6fb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:28.157461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76d525-edb7-445f-9954-f0c0dde6fb07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:28.174987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76d525-edb7-445f-9954-f0c0dde6fb07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-0c76d525-edb7-445f-9954-f0c0dde6fb07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:28:28.179971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:28.181277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:28.182528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 610/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:28:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:28.188345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511bd63-b074-40d6-8515-d8d09aa0d2eb None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:28.330433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511bd63-b074-40d6-8515-d8d09aa0d2eb None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:28.370256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:28.377154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5fc94-6468-4f4a-8539-ea0d8813c33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:28.377825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5fc94-6468-4f4a-8539-ea0d8813c33d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:28.378146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5fc94-6468-4f4a-8539-ea0d8813c33d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:28.378693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5fc94-6468-4f4a-8539-ea0d8813c33d None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:28.379812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5fc94-6468-4f4a-8539-ea0d8813c33d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:28.379812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5fc94-6468-4f4a-8539-ea0d8813c33d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:28.383393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5fc94-6468-4f4a-8539-ea0d8813c33d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:28.398505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 640/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:28:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:28.407221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76d525-edb7-445f-9954-f0c0dde6fb07 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:28.509675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:28.535816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76d525-edb7-445f-9954-f0c0dde6fb07 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:28.587004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99c54d-d040-4992-9d37-493f802d79ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:28.588062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99c54d-d040-4992-9d37-493f802d79ef None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:28.588718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99c54d-d040-4992-9d37-493f802d79ef None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:28.589357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99c54d-d040-4992-9d37-493f802d79ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:28.604724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99c54d-d040-4992-9d37-493f802d79ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:28.606922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99c54d-d040-4992-9d37-493f802d79ef None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-1a99c54d-d040-4992-9d37-493f802d79ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:28.624148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:28.625503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:28.626821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:28.626821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:28.626821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:28.626821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 611/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:28.635756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5fc94-6468-4f4a-8539-ea0d8813c33d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:28.799639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5fc94-6468-4f4a-8539-ea0d8813c33d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:28.848001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7bae9-77b6-45d5-8672-d85c77600a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:28.848856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7bae9-77b6-45d5-8672-d85c77600a79 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:28.849326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7bae9-77b6-45d5-8672-d85c77600a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:28.849667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7bae9-77b6-45d5-8672-d85c77600a79 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:28.850488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7bae9-77b6-45d5-8672-d85c77600a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:28.850818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7bae9-77b6-45d5-8672-d85c77600a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:28.854688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7bae9-77b6-45d5-8672-d85c77600a79 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:28.872527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 641/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:28.881350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99c54d-d040-4992-9d37-493f802d79ef None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:29.017862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99c54d-d040-4992-9d37-493f802d79ef None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:29.058494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:29.069150 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdccae-907a-4437-b835-bf42038d9ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:29.069825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdccae-907a-4437-b835-bf42038d9ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:29.070115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdccae-907a-4437-b835-bf42038d9ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:29.070353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdccae-907a-4437-b835-bf42038d9ce9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:29.082097 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdccae-907a-4437-b835-bf42038d9ce9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:29.083649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdccae-907a-4437-b835-bf42038d9ce9 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-80fdccae-907a-4437-b835-bf42038d9ce9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:28:29.097340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:29.099982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:29.102095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:29.102095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:29.102095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:28:29.102095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:28:29.102095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:28:29.102095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:28:29.102095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:28:29.102095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:28:29.102095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 08:28:29.102095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:29.102095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 612/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:29.105575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7bae9-77b6-45d5-8672-d85c77600a79 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:29.213369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:29.288862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7bae9-77b6-45d5-8672-d85c77600a79 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:29.336074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ede30-d50c-4272-894b-705a4b9d754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:29.336844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ede30-d50c-4272-894b-705a4b9d754d None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:29.337066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ede30-d50c-4272-894b-705a4b9d754d None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:29.337277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ede30-d50c-4272-894b-705a4b9d754d None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:29.338179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ede30-d50c-4272-894b-705a4b9d754d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:29.338436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ede30-d50c-4272-894b-705a4b9d754d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:29.340539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ede30-d50c-4272-894b-705a4b9d754d None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:29.362451 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 642/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 08:28:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:29.369128 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdccae-907a-4437-b835-bf42038d9ce9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:29.533440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdccae-907a-4437-b835-bf42038d9ce9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:29.585349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255248ae-d98e-4f4f-ac45-3a4e5876a242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:29.585992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255248ae-d98e-4f4f-ac45-3a4e5876a242 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:29.586284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255248ae-d98e-4f4f-ac45-3a4e5876a242 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:29.586546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255248ae-d98e-4f4f-ac45-3a4e5876a242 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:29.659684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:29.678547 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255248ae-d98e-4f4f-ac45-3a4e5876a242 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:29.680540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255248ae-d98e-4f4f-ac45-3a4e5876a242 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:29.807422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 613/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:28:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 08:28:29.814470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ede30-d50c-4272-894b-705a4b9d754d None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:29.937356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ede30-d50c-4272-894b-705a4b9d754d None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:29.952904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:29.978433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ea685-6f72-4e74-967b-9e2dc96f5a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:29.979095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ea685-6f72-4e74-967b-9e2dc96f5a84 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:29.979373 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ea685-6f72-4e74-967b-9e2dc96f5a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:29.979646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ea685-6f72-4e74-967b-9e2dc96f5a84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:29.991765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ea685-6f72-4e74-967b-9e2dc96f5a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:29.993274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ea685-6f72-4e74-967b-9e2dc96f5a84 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-d04ea685-6f72-4e74-967b-9e2dc96f5a84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:30.013712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:30.015352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:30.016827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:30.016827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:30.016827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:30.016827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 643/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:30.023758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255248ae-d98e-4f4f-ac45-3a4e5876a242 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:30.136709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:30.157213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255248ae-d98e-4f4f-ac45-3a4e5876a242 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:30.212474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0840b2-a128-4954-8e77-9ea1373df4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:30.213336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0840b2-a128-4954-8e77-9ea1373df4ad None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:30.213484 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0840b2-a128-4954-8e77-9ea1373df4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:30.213862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0840b2-a128-4954-8e77-9ea1373df4ad None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:30.214637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0840b2-a128-4954-8e77-9ea1373df4ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:30.215608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0840b2-a128-4954-8e77-9ea1373df4ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:30.217596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0840b2-a128-4954-8e77-9ea1373df4ad None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:30.233851 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 614/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:28:30.239966 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ea685-6f72-4e74-967b-9e2dc96f5a84 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:30.371075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ea685-6f72-4e74-967b-9e2dc96f5a84 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:30.412593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71220d47-f5e1-4ce7-b379-322c07f9928e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:30.413256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71220d47-f5e1-4ce7-b379-322c07f9928e None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:30.413530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71220d47-f5e1-4ce7-b379-322c07f9928e None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:30.413799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71220d47-f5e1-4ce7-b379-322c07f9928e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:30.422106 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71220d47-f5e1-4ce7-b379-322c07f9928e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:30.423557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71220d47-f5e1-4ce7-b379-322c07f9928e None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-71220d47-f5e1-4ce7-b379-322c07f9928e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:28:30.438359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:30.439453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:30.440619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:28:30.440619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:28:30.440619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:28:30.440619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:28:30.440619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:28:30.440619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:28:30.440619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:28:30.440619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:30.440619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 644/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:28:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:30.447516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0840b2-a128-4954-8e77-9ea1373df4ad None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:30.586338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0840b2-a128-4954-8e77-9ea1373df4ad None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:30.646656 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51461e02-1c02-410d-a563-122931cd689d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:30.647673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51461e02-1c02-410d-a563-122931cd689d None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:30.648171 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51461e02-1c02-410d-a563-122931cd689d None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:30.648668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51461e02-1c02-410d-a563-122931cd689d None admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:30.650002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51461e02-1c02-410d-a563-122931cd689d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:30.650558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51461e02-1c02-410d-a563-122931cd689d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:30.654054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51461e02-1c02-410d-a563-122931cd689d None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:30.676829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 615/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:28:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:30.683036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71220d47-f5e1-4ce7-b379-322c07f9928e None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:30.718277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:30.796567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71220d47-f5e1-4ce7-b379-322c07f9928e None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:30.832094 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726b542-949e-4d91-9a8c-9f0ae8ea4c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:30.832818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726b542-949e-4d91-9a8c-9f0ae8ea4c30 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:30.833132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726b542-949e-4d91-9a8c-9f0ae8ea4c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:30.833474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726b542-949e-4d91-9a8c-9f0ae8ea4c30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:30.834365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726b542-949e-4d91-9a8c-9f0ae8ea4c30 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=6b40ee263df74e58bf3f958c01eb742f) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:30.834751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726b542-949e-4d91-9a8c-9f0ae8ea4c30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:30.836436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726b542-949e-4d91-9a8c-9f0ae8ea4c30 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:30.843508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 645/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:28:30 2022] GET /identity/v3/role_assignments?user.id=6b40ee263df74e58bf3f958c01eb742f&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:28:33.414302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b7bc5-8506-4f8a-8378-1772cb81f24d None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:33.414846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b7bc5-8506-4f8a-8378-1772cb81f24d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:33.414846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b7bc5-8506-4f8a-8378-1772cb81f24d None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:33.416189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 616/1261] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:28:33.422687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07407f-fc5f-42dd-954d-15c04f5f4128 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:33.423223 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07407f-fc5f-42dd-954d-15c04f5f4128 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:33.423223 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07407f-fc5f-42dd-954d-15c04f5f4128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:33.443173 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-4e07407f-fc5f-42dd-954d-15c04f5f4128 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:28:33.444407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4e07407f-fc5f-42dd-954d-15c04f5f4128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:28:33.520630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 646/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:28:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:28:33.793576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b7bc5-8506-4f8a-8378-1772cb81f24d None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:33.805732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:33.908032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b7bc5-8506-4f8a-8378-1772cb81f24d None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:33.945981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da831003-6f90-4550-b3fb-983dc3ff11f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:33.946735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da831003-6f90-4550-b3fb-983dc3ff11f9 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:33.947044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da831003-6f90-4550-b3fb-983dc3ff11f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:33.947362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da831003-6f90-4550-b3fb-983dc3ff11f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:33.959746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da831003-6f90-4550-b3fb-983dc3ff11f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-da831003-6f90-4550-b3fb-983dc3ff11f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:28:33.964620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:33.965747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:33.966838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 617/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:28:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:28:33.973382 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07407f-fc5f-42dd-954d-15c04f5f4128 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:33.997885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:34.085358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07407f-fc5f-42dd-954d-15c04f5f4128 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:34.128832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0e286-cbf6-40fc-a3e7-28fe6c1bbc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:34.129633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0e286-cbf6-40fc-a3e7-28fe6c1bbc17 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:34.129952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0e286-cbf6-40fc-a3e7-28fe6c1bbc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:34.130328 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0e286-cbf6-40fc-a3e7-28fe6c1bbc17 None admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:34.131280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0e286-cbf6-40fc-a3e7-28fe6c1bbc17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:34.131604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0e286-cbf6-40fc-a3e7-28fe6c1bbc17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:34.133434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0e286-cbf6-40fc-a3e7-28fe6c1bbc17 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:34.150376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 647/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:28:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:34.151768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da831003-6f90-4550-b3fb-983dc3ff11f9 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:34.231283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:34.267021 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da831003-6f90-4550-b3fb-983dc3ff11f9 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:34.304201 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb221a-fb7c-4f38-b5fb-16a99d1386ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:34.304843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb221a-fb7c-4f38-b5fb-16a99d1386ec None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:34.305155 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb221a-fb7c-4f38-b5fb-16a99d1386ec None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:34.305432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb221a-fb7c-4f38-b5fb-16a99d1386ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:34.316801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb221a-fb7c-4f38-b5fb-16a99d1386ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:34.318287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb221a-fb7c-4f38-b5fb-16a99d1386ec None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-93cb221a-fb7c-4f38-b5fb-16a99d1386ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:34.333034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:34.334239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:34.335413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:34.335413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:34.335413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:34.335413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 618/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:34.340043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0e286-cbf6-40fc-a3e7-28fe6c1bbc17 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:34.431284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:34.451621 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0e286-cbf6-40fc-a3e7-28fe6c1bbc17 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:34.487101 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239e238-dc90-409c-851a-a429d260ec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:34.487827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239e238-dc90-409c-851a-a429d260ec01 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:34.488107 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239e238-dc90-409c-851a-a429d260ec01 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:34.488379 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239e238-dc90-409c-851a-a429d260ec01 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:34.489027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239e238-dc90-409c-851a-a429d260ec01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:34.489396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239e238-dc90-409c-851a-a429d260ec01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:34.490886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239e238-dc90-409c-851a-a429d260ec01 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:34.504888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 648/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:34.513746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb221a-fb7c-4f38-b5fb-16a99d1386ec None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:34.639337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb221a-fb7c-4f38-b5fb-16a99d1386ec None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:34.663123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:34.678933 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41427f73-1f7e-43f7-a705-f040fb5ba31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:34.680312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41427f73-1f7e-43f7-a705-f040fb5ba31a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:34.680312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41427f73-1f7e-43f7-a705-f040fb5ba31a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:34.680312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41427f73-1f7e-43f7-a705-f040fb5ba31a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:34.688372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41427f73-1f7e-43f7-a705-f040fb5ba31a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:34.689854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41427f73-1f7e-43f7-a705-f040fb5ba31a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-41427f73-1f7e-43f7-a705-f040fb5ba31a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:28:34.701389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:34.703112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:34.704926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:34.704926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:34.704926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:28:34.704926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:28:34.704926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:28:34.704926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:28:34.704926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:28:34.704926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:28:34.704926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 08:28:34.704926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:34.704926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 619/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:28:34.709944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239e238-dc90-409c-851a-a429d260ec01 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:34.814688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239e238-dc90-409c-851a-a429d260ec01 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:34.850884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe35abf-856c-42e4-99f4-683c14bedfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:34.851499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe35abf-856c-42e4-99f4-683c14bedfbd None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:34.851780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe35abf-856c-42e4-99f4-683c14bedfbd None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:34.852051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe35abf-856c-42e4-99f4-683c14bedfbd None admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:34.852793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe35abf-856c-42e4-99f4-683c14bedfbd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:34.853123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe35abf-856c-42e4-99f4-683c14bedfbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:34.854574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe35abf-856c-42e4-99f4-683c14bedfbd None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:34.865036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:34.874665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 649/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 08:28:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:34.880764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41427f73-1f7e-43f7-a705-f040fb5ba31a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:34.993391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41427f73-1f7e-43f7-a705-f040fb5ba31a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:35.029829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd80537-9e9e-4ea8-8992-3f02874b3340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:35.030479 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd80537-9e9e-4ea8-8992-3f02874b3340 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:35.030757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd80537-9e9e-4ea8-8992-3f02874b3340 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:35.031028 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd80537-9e9e-4ea8-8992-3f02874b3340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:35.043025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd80537-9e9e-4ea8-8992-3f02874b3340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:35.044494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd80537-9e9e-4ea8-8992-3f02874b3340 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-1dd80537-9e9e-4ea8-8992-3f02874b3340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:35.060054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:35.061297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:35.062604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:35.062604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:35.062604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:35.062604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 620/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:35.067255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe35abf-856c-42e4-99f4-683c14bedfbd None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:35.180787 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe35abf-856c-42e4-99f4-683c14bedfbd None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:35.217184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd3a00-cd28-4715-8f70-cd3c61686f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:35.217804 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd3a00-cd28-4715-8f70-cd3c61686f48 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:35.218080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd3a00-cd28-4715-8f70-cd3c61686f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:35.218348 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd3a00-cd28-4715-8f70-cd3c61686f48 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:35.218993 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd3a00-cd28-4715-8f70-cd3c61686f48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:35.219292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd3a00-cd28-4715-8f70-cd3c61686f48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:35.220649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd3a00-cd28-4715-8f70-cd3c61686f48 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:35.232783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 650/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:35.238604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd80537-9e9e-4ea8-8992-3f02874b3340 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:35.272441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:35.346703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd80537-9e9e-4ea8-8992-3f02874b3340 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:35.382285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a25c3-0248-47b3-9eed-94c761833869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:35.382907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a25c3-0248-47b3-9eed-94c761833869 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:35.383185 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a25c3-0248-47b3-9eed-94c761833869 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:35.383451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a25c3-0248-47b3-9eed-94c761833869 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:35.391133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a25c3-0248-47b3-9eed-94c761833869 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:35.392577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a25c3-0248-47b3-9eed-94c761833869 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-5f9a25c3-0248-47b3-9eed-94c761833869 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:28:35.404011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:35.405657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:35.406751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:28:35.406751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:28:35.406751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:28:35.406751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:28:35.406751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:28:35.406751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:28:35.406751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:28:35.406751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:35.406751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 621/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:28:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:35.411345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd3a00-cd28-4715-8f70-cd3c61686f48 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:35.473467 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:35.516792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd3a00-cd28-4715-8f70-cd3c61686f48 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:35.552306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009918dd-577f-4e7d-8c6f-1678bed64ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:35.552921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009918dd-577f-4e7d-8c6f-1678bed64ded None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:35.553231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009918dd-577f-4e7d-8c6f-1678bed64ded None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:35.553505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009918dd-577f-4e7d-8c6f-1678bed64ded None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:35.554198 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009918dd-577f-4e7d-8c6f-1678bed64ded None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:35.554505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009918dd-577f-4e7d-8c6f-1678bed64ded None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:35.555925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009918dd-577f-4e7d-8c6f-1678bed64ded None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:35.568175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 651/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:28:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:35.574836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a25c3-0248-47b3-9eed-94c761833869 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:35.670487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:35.679346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a25c3-0248-47b3-9eed-94c761833869 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:35.713351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d34af-de93-48db-922e-8df32de2dc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:35.714002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d34af-de93-48db-922e-8df32de2dc17 None admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:35.714280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d34af-de93-48db-922e-8df32de2dc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:35.714554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d34af-de93-48db-922e-8df32de2dc17 None admin] PATH_INFO: `/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/6b40ee263df74e58bf3f958c01eb742f/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:35.716488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d34af-de93-48db-922e-8df32de2dc17 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca4d4801f2f40419f6881984fbbfbcd, user_id=6b40ee263df74e58bf3f958c01eb742f, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:35.717944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d34af-de93-48db-922e-8df32de2dc17 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:35.731606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 622/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 08:28:35 2022] PUT /identity/v3/projects/9ca4d4801f2f40419f6881984fbbfbcd/users/6b40ee263df74e58bf3f958c01eb742f/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:28:37.969363 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8efd01-e39a-4ef9-9675-132d9c931e5e None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:37.969934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8efd01-e39a-4ef9-9675-132d9c931e5e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:37.969934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8efd01-e39a-4ef9-9675-132d9c931e5e None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:37.971235 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 652/1274] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:28:37.976674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b36ba-19f8-4287-9ca2-41b981bc8e27 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:37.977172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b36ba-19f8-4287-9ca2-41b981bc8e27 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:37.977283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b36ba-19f8-4287-9ca2-41b981bc8e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:37.995862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-f91b36ba-19f8-4287-9ca2-41b981bc8e27 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:28:37.997675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f91b36ba-19f8-4287-9ca2-41b981bc8e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:28:38.067438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 623/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:28:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:28:38.358899 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8efd01-e39a-4ef9-9675-132d9c931e5e None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:38.468323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8efd01-e39a-4ef9-9675-132d9c931e5e None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:38.486787 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:38.505186 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62792255-4a28-46aa-8903-5669d4ad51d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:38.505824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62792255-4a28-46aa-8903-5669d4ad51d4 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:38.506105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62792255-4a28-46aa-8903-5669d4ad51d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:38.506382 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62792255-4a28-46aa-8903-5669d4ad51d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:38.518346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62792255-4a28-46aa-8903-5669d4ad51d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-62792255-4a28-46aa-8903-5669d4ad51d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 08:28:38.523539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:38.525334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:38.528365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 653/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 08:28:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:28:38.531964 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b36ba-19f8-4287-9ca2-41b981bc8e27 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:38.626113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:38.646934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b36ba-19f8-4287-9ca2-41b981bc8e27 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:38.684660 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c669b3-049d-4324-922d-c32c0712eb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:38.685312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c669b3-049d-4324-922d-c32c0712eb34 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:38.685593 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c669b3-049d-4324-922d-c32c0712eb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:38.685863 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c669b3-049d-4324-922d-c32c0712eb34 None admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:38.686548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c669b3-049d-4324-922d-c32c0712eb34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:38.686847 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c669b3-049d-4324-922d-c32c0712eb34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:38.688252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c669b3-049d-4324-922d-c32c0712eb34 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:38.697477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 624/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 08:28:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:38.704266 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62792255-4a28-46aa-8903-5669d4ad51d4 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:38.813873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62792255-4a28-46aa-8903-5669d4ad51d4 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:38.850893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec3723-f5b2-421f-b0b7-136c909585ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:38.851530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec3723-f5b2-421f-b0b7-136c909585ac None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:38.851809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec3723-f5b2-421f-b0b7-136c909585ac None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:38.852081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec3723-f5b2-421f-b0b7-136c909585ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:38.862917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec3723-f5b2-421f-b0b7-136c909585ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:38.864376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec3723-f5b2-421f-b0b7-136c909585ac None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-d6ec3723-f5b2-421f-b0b7-136c909585ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:38.879949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:38.885948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:38.887390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:38.887390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:38.887390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:38.887390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 654/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 08:28:38.887617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c669b3-049d-4324-922d-c32c0712eb34 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:38.989532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c669b3-049d-4324-922d-c32c0712eb34 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:39.023172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3eb900-ab0e-4f4b-8f09-46c489d599de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:39.023769 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3eb900-ab0e-4f4b-8f09-46c489d599de None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:39.024051 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3eb900-ab0e-4f4b-8f09-46c489d599de None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:39.024326 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3eb900-ab0e-4f4b-8f09-46c489d599de None admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:39.024961 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3eb900-ab0e-4f4b-8f09-46c489d599de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:39.025293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3eb900-ab0e-4f4b-8f09-46c489d599de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:39.026665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3eb900-ab0e-4f4b-8f09-46c489d599de None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:39.039344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 625/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:39.045249 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec3723-f5b2-421f-b0b7-136c909585ac None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:39.076675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:39.155350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec3723-f5b2-421f-b0b7-136c909585ac None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:39.189653 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571d377-f01e-4145-bea1-6fa89d5f5e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:39.190284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571d377-f01e-4145-bea1-6fa89d5f5e86 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:39.190554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571d377-f01e-4145-bea1-6fa89d5f5e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:39.190821 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571d377-f01e-4145-bea1-6fa89d5f5e86 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:39.198544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571d377-f01e-4145-bea1-6fa89d5f5e86 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:39.200606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a571d377-f01e-4145-bea1-6fa89d5f5e86 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-a571d377-f01e-4145-bea1-6fa89d5f5e86 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 08:28:39.211930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:39.213078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:39.214461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:39.214461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:39.214461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 08:28:39.214461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 08:28:39.214461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 08:28:39.214461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 08:28:39.214461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 08:28:39.214461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 08:28:39.214461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 08:28:39.214461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:39.214461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 655/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:39.219465 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3eb900-ab0e-4f4b-8f09-46c489d599de None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:39.223624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:39.328236 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3eb900-ab0e-4f4b-8f09-46c489d599de None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:39.363411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bd866-42c2-4acf-adf1-df3e4d67b947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:39.364015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bd866-42c2-4acf-adf1-df3e4d67b947 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:39.364291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bd866-42c2-4acf-adf1-df3e4d67b947 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:39.364563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bd866-42c2-4acf-adf1-df3e4d67b947 None admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:39.365325 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bd866-42c2-4acf-adf1-df3e4d67b947 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:39.365631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bd866-42c2-4acf-adf1-df3e4d67b947 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:39.367078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bd866-42c2-4acf-adf1-df3e4d67b947 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:39.386663 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 626/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 08:28:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:39.392464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571d377-f01e-4145-bea1-6fa89d5f5e86 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:39.486871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:39.498017 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571d377-f01e-4145-bea1-6fa89d5f5e86 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:39.535491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af09ba3-8ef7-462c-a2da-8e9d951e75c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:39.536221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af09ba3-8ef7-462c-a2da-8e9d951e75c3 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:39.536571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af09ba3-8ef7-462c-a2da-8e9d951e75c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:39.536904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af09ba3-8ef7-462c-a2da-8e9d951e75c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:39.612472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af09ba3-8ef7-462c-a2da-8e9d951e75c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:39.621284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af09ba3-8ef7-462c-a2da-8e9d951e75c3 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:39.712646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:39.723874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 656/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 08:28:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 08:28:39.729784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bd866-42c2-4acf-adf1-df3e4d67b947 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:39.791829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:39.831396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bd866-42c2-4acf-adf1-df3e4d67b947 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:39.869204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f783ea-fccc-46e7-bd3c-371605311a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:39.869865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f783ea-fccc-46e7-bd3c-371605311a1a None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:39.870185 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f783ea-fccc-46e7-bd3c-371605311a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:39.870464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f783ea-fccc-46e7-bd3c-371605311a1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:39.882029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f783ea-fccc-46e7-bd3c-371605311a1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:39.883451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f783ea-fccc-46e7-bd3c-371605311a1a None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-a5f783ea-fccc-46e7-bd3c-371605311a1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 08:28:39.900923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:39.902830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 08:28:39.904796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 08:28:39.904796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 08:28:39.904796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:39.904796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 627/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:28:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:39.908881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af09ba3-8ef7-462c-a2da-8e9d951e75c3 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:40.026080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af09ba3-8ef7-462c-a2da-8e9d951e75c3 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:40.061398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c8573-8f87-457b-82f5-de093897c643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:40.062021 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c8573-8f87-457b-82f5-de093897c643 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:40.062190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c8573-8f87-457b-82f5-de093897c643 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:40.062462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c8573-8f87-457b-82f5-de093897c643 None admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:40.063211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c8573-8f87-457b-82f5-de093897c643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:40.063528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c8573-8f87-457b-82f5-de093897c643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:40.064885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c8573-8f87-457b-82f5-de093897c643 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:40.077340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 657/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 08:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:40.082960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f783ea-fccc-46e7-bd3c-371605311a1a None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:40.186366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f783ea-fccc-46e7-bd3c-371605311a1a None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:40.200049 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:40.221451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a14070-50a2-4c61-acdc-d9769cc8df43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:40.222078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a14070-50a2-4c61-acdc-d9769cc8df43 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:40.222358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a14070-50a2-4c61-acdc-d9769cc8df43 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:40.222633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a14070-50a2-4c61-acdc-d9769cc8df43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:40.229353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a14070-50a2-4c61-acdc-d9769cc8df43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:40.231298 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a14070-50a2-4c61-acdc-d9769cc8df43 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application [None req-01a14070-50a2-4c61-acdc-d9769cc8df43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 08:28:40.244718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:40.248363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:40.250078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 08:28:40.250078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 08:28:40.250078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 08:28:40.250078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 08:28:40.250078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 08:28:40.250078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 08:28:40.250078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 08:28:40.250078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: ERROR keystone.server.flask.application Sep 28 08:28:40.250078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 628/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 08:28:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:40.256240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c8573-8f87-457b-82f5-de093897c643 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:40.309450 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:40.359609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c8573-8f87-457b-82f5-de093897c643 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:40.392486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57249ca-c0b5-405f-852c-ea0d53352d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:40.393105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57249ca-c0b5-405f-852c-ea0d53352d71 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:40.393399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57249ca-c0b5-405f-852c-ea0d53352d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:40.393672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57249ca-c0b5-405f-852c-ea0d53352d71 None admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:40.394360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57249ca-c0b5-405f-852c-ea0d53352d71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:40.394667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57249ca-c0b5-405f-852c-ea0d53352d71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:40.396072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57249ca-c0b5-405f-852c-ea0d53352d71 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:40.408100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 658/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 08:28:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:40.414301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a14070-50a2-4c61-acdc-d9769cc8df43 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:40.514136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a14070-50a2-4c61-acdc-d9769cc8df43 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:40.554884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04249087-2421-4779-93cd-f3b3cf11f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:40.555490 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04249087-2421-4779-93cd-f3b3cf11f3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:40.555768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04249087-2421-4779-93cd-f3b3cf11f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:40.556041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04249087-2421-4779-93cd-f3b3cf11f3f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:40.556756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04249087-2421-4779-93cd-f3b3cf11f3f5 None admin] RBAC: Adding query filter params (role.id=9c7a8295343a410f8b835c8f65408be6, scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd, user.id=6b40ee263df74e58bf3f958c01eb742f) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:40.557065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04249087-2421-4779-93cd-f3b3cf11f3f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:40.559097 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04249087-2421-4779-93cd-f3b3cf11f3f5 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:40.564786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 629/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 08:28:40 2022] GET /identity/v3/role_assignments?user.id=6b40ee263df74e58bf3f958c01eb742f&scope.project.id=9ca4d4801f2f40419f6881984fbbfbcd&role.id=9c7a8295343a410f8b835c8f65408be6 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:42.760067 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836a6a3-fe9e-4ab0-aa5c-72a4f9b21bf2 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:42.760814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836a6a3-fe9e-4ab0-aa5c-72a4f9b21bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:42.760814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836a6a3-fe9e-4ab0-aa5c-72a4f9b21bf2 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:42.762528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 659/1288] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:28:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:28:42.768150 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d9a46-9031-46ee-b1cd-866860ecdacb None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:42.768687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d9a46-9031-46ee-b1cd-866860ecdacb None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:42.768687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d9a46-9031-46ee-b1cd-866860ecdacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:42.790596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-e04d9a46-9031-46ee-b1cd-866860ecdacb None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:28:42.791700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e04d9a46-9031-46ee-b1cd-866860ecdacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:28:42.822128 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:42.861541 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 630/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:28:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 08:28:43.138615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836a6a3-fe9e-4ab0-aa5c-72a4f9b21bf2 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:43.239655 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836a6a3-fe9e-4ab0-aa5c-72a4f9b21bf2 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:43.251359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:43.297767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2b6ff-8cae-412f-8043-41db8e40a4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:43.298403 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2b6ff-8cae-412f-8043-41db8e40a4be None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:43.298748 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2b6ff-8cae-412f-8043-41db8e40a4be None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:43.299018 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2b6ff-8cae-412f-8043-41db8e40a4be None admin] PATH_INFO: `/v3/services/placement` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:43.299601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2b6ff-8cae-412f-8043-41db8e40a4be None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:43.300948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2b6ff-8cae-412f-8043-41db8e40a4be None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-f2c2b6ff-8cae-412f-8043-41db8e40a4be None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:43.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:43.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:43.314414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 660/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 08:28:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:43.320403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d9a46-9031-46ee-b1cd-866860ecdacb None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:43.420831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d9a46-9031-46ee-b1cd-866860ecdacb None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:43.455202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8044608-0d89-4c79-b79b-5cb9a937a725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:43.455803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8044608-0d89-4c79-b79b-5cb9a937a725 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:43.456088 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8044608-0d89-4c79-b79b-5cb9a937a725 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:43.456364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8044608-0d89-4c79-b79b-5cb9a937a725 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:43.456991 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8044608-0d89-4c79-b79b-5cb9a937a725 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:43.457322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8044608-0d89-4c79-b79b-5cb9a937a725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:43.458687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8044608-0d89-4c79-b79b-5cb9a937a725 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:43.463716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 631/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Sep 28 08:28:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:43.469581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2b6ff-8cae-412f-8043-41db8e40a4be None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:43.578097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2b6ff-8cae-412f-8043-41db8e40a4be None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:43.613900 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15e23c-fa42-463d-88bf-a0fc0c307015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:43.614508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15e23c-fa42-463d-88bf-a0fc0c307015 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:43.614785 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15e23c-fa42-463d-88bf-a0fc0c307015 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:43.615055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15e23c-fa42-463d-88bf-a0fc0c307015 None admin] PATH_INFO: `/v3/services` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:43.615678 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15e23c-fa42-463d-88bf-a0fc0c307015 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:43.615979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15e23c-fa42-463d-88bf-a0fc0c307015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:43.617357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15e23c-fa42-463d-88bf-a0fc0c307015 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:43.621939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 661/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Sep 28 08:28:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:45.806538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f044408-36fa-4b36-814f-b71d2bd1d556 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:45.807036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f044408-36fa-4b36-814f-b71d2bd1d556 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:45.807036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f044408-36fa-4b36-814f-b71d2bd1d556 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:45.808375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 632/1293] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:28:45.813532 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5131d-de2a-4815-85a0-6b836d59bb30 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:45.813901 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5131d-de2a-4815-85a0-6b836d59bb30 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:45.814079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5131d-de2a-4815-85a0-6b836d59bb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:45.832291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-43e5131d-de2a-4815-85a0-6b836d59bb30 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:28:45.833580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-43e5131d-de2a-4815-85a0-6b836d59bb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:28:45.864759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:45.903473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 662/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:28:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:28:46.192758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f044408-36fa-4b36-814f-b71d2bd1d556 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:46.224346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:46.294618 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f044408-36fa-4b36-814f-b71d2bd1d556 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:46.327607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4789e5-4ec0-46eb-ad49-a5ed097342c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:46.328400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4789e5-4ec0-46eb-ad49-a5ed097342c5 None admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:46.329399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4789e5-4ec0-46eb-ad49-a5ed097342c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:46.329681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4789e5-4ec0-46eb-ad49-a5ed097342c5 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:46.330332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4789e5-4ec0-46eb-ad49-a5ed097342c5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:46.331695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4789e5-4ec0-46eb-ad49-a5ed097342c5 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:46.346320 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 633/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 08:28:46 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:28:48.581037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377132-c135-4296-8171-fad62d5def15 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:48.581633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377132-c135-4296-8171-fad62d5def15 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:48.581633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d377132-c135-4296-8171-fad62d5def15 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:48.582964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 663/1296] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:28:48.588496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa856d-34d9-4dc8-9dd3-126e204bfa70 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:48.589013 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa856d-34d9-4dc8-9dd3-126e204bfa70 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:48.589013 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa856d-34d9-4dc8-9dd3-126e204bfa70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:48.607754 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ccfa856d-34d9-4dc8-9dd3-126e204bfa70 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:28:48.608942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ccfa856d-34d9-4dc8-9dd3-126e204bfa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:28:48.683426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 634/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:28:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:28:48.946442 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d377132-c135-4296-8171-fad62d5def15 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:49.050704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d377132-c135-4296-8171-fad62d5def15 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:49.087645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda41bb5-0658-4721-81cc-8885e05a9a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:49.088284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda41bb5-0658-4721-81cc-8885e05a9a82 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:49.088564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda41bb5-0658-4721-81cc-8885e05a9a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:49.088840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda41bb5-0658-4721-81cc-8885e05a9a82 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:49.089505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda41bb5-0658-4721-81cc-8885e05a9a82 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:49.090864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda41bb5-0658-4721-81cc-8885e05a9a82 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-eda41bb5-0658-4721-81cc-8885e05a9a82 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:49.107362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:49.109310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:49.110888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 664/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 08:28:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:49.114588 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa856d-34d9-4dc8-9dd3-126e204bfa70 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:49.139955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:49.244606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa856d-34d9-4dc8-9dd3-126e204bfa70 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:49.280491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a9230-c30f-4e41-9a5f-643a6ea99275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:49.281117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a9230-c30f-4e41-9a5f-643a6ea99275 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:49.281409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a9230-c30f-4e41-9a5f-643a6ea99275 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:49.281680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a9230-c30f-4e41-9a5f-643a6ea99275 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:49.282314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a9230-c30f-4e41-9a5f-643a6ea99275 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:49.282620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a9230-c30f-4e41-9a5f-643a6ea99275 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:49.283983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a9230-c30f-4e41-9a5f-643a6ea99275 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:49.289381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 635/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Sep 28 08:28:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:49.296480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda41bb5-0658-4721-81cc-8885e05a9a82 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:49.333697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:49.406615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda41bb5-0658-4721-81cc-8885e05a9a82 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:49.449493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d1455-c07b-48d5-9311-d96f391cbcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:49.450125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1455-c07b-48d5-9311-d96f391cbcb6 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:49.450408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1455-c07b-48d5-9311-d96f391cbcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:49.450684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1455-c07b-48d5-9311-d96f391cbcb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:49.451401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d1455-c07b-48d5-9311-d96f391cbcb6 None admin] RBAC: Adding query filter params (interface=public, service_id=51b26f1711be4efbb52a4edb49abf7ea, region_id=RegionOne) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:49.451720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d1455-c07b-48d5-9311-d96f391cbcb6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:49.453122 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d1455-c07b-48d5-9311-d96f391cbcb6 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:49.459006 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 665/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 08:28:49 2022] GET /identity/v3/endpoints?service_id=51b26f1711be4efbb52a4edb49abf7ea&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:49.464994 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a9230-c30f-4e41-9a5f-643a6ea99275 None admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:49.552523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:49.572056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a9230-c30f-4e41-9a5f-643a6ea99275 None admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:49.609675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a4f71-b13c-46e1-9548-18c1b1800b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:49.610341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a4f71-b13c-46e1-9548-18c1b1800b55 None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:49.610560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a4f71-b13c-46e1-9548-18c1b1800b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:49.610825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a4f71-b13c-46e1-9548-18c1b1800b55 None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:49.611440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a4f71-b13c-46e1-9548-18c1b1800b55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:49.612817 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a4f71-b13c-46e1-9548-18c1b1800b55 None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:49.620283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 636/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 08:28:49 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:28:51.779554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c303e74-f658-4495-b155-3b3cd59e71da None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:51.780345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c303e74-f658-4495-b155-3b3cd59e71da None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:51.780345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c303e74-f658-4495-b155-3b3cd59e71da None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:51.781839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 666/1302] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:28:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:28:51.787705 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd04a6-dbc1-44ef-8072-08a9f2af9400 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:51.788241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd04a6-dbc1-44ef-8072-08a9f2af9400 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:51.788241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd04a6-dbc1-44ef-8072-08a9f2af9400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:51.806105 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-8dfd04a6-dbc1-44ef-8072-08a9f2af9400 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:28:51.807276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8dfd04a6-dbc1-44ef-8072-08a9f2af9400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:28:51.872089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 637/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:28:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 08:28:52.139579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c303e74-f658-4495-b155-3b3cd59e71da None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:52.238144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:52.245332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c303e74-f658-4495-b155-3b3cd59e71da None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:52.278846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecaf0d-97e4-4cb0-9b35-53ea28819e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:52.279473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecaf0d-97e4-4cb0-9b35-53ea28819e72 None admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:52.280007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecaf0d-97e4-4cb0-9b35-53ea28819e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:52.280278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecaf0d-97e4-4cb0-9b35-53ea28819e72 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:52.280851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adecaf0d-97e4-4cb0-9b35-53ea28819e72 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:52.282231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adecaf0d-97e4-4cb0-9b35-53ea28819e72 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application [None req-adecaf0d-97e4-4cb0-9b35-53ea28819e72 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 08:28:52.290849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 08:28:52.292014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application with Lock( Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self._enter() Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application return self.creator() Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application created_value = creator( Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: ERROR keystone.server.flask.application Sep 28 08:28:52.293721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 667/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 08:28:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 08:28:52.298487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd04a6-dbc1-44ef-8072-08a9f2af9400 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:52.376948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:28:52.405939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd04a6-dbc1-44ef-8072-08a9f2af9400 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:52.440076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3badc-4a91-411b-90a7-3237406120aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:52.440675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3badc-4a91-411b-90a7-3237406120aa None admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:52.440956 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3badc-4a91-411b-90a7-3237406120aa None admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:52.441258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3badc-4a91-411b-90a7-3237406120aa None admin] PATH_INFO: `/v3/services` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:52.441885 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3badc-4a91-411b-90a7-3237406120aa None admin] RBAC: Adding query filter params (name=placement) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:28:52.442191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3badc-4a91-411b-90a7-3237406120aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:52.443552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3badc-4a91-411b-90a7-3237406120aa None admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:52.448562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 638/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Sep 28 08:28:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:28:52.454458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecaf0d-97e4-4cb0-9b35-53ea28819e72 None admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:28:52.557170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecaf0d-97e4-4cb0-9b35-53ea28819e72 None admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:28:52.590883 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08edf27-4d4d-43ff-ab29-d960b13f2d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:28:52.591487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08edf27-4d4d-43ff-ab29-d960b13f2d31 None admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:28:52.591761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08edf27-4d4d-43ff-ab29-d960b13f2d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:28:52.592028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08edf27-4d4d-43ff-ab29-d960b13f2d31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:28:52.592700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08edf27-4d4d-43ff-ab29-d960b13f2d31 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:28:52.594072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08edf27-4d4d-43ff-ab29-d960b13f2d31 None admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:28:52.613887 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 668/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:28:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:29:02.445554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=30 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:29:02.609362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:29:57.745426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfea910-558f-4db3-8695-a7bf2b0aa470 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:29:57.745871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfea910-558f-4db3-8695-a7bf2b0aa470 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:29:57.746040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfea910-558f-4db3-8695-a7bf2b0aa470 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:29:57.752486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce6782-c7ac-4853-9f09-f9211a581ad6 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:29:57.753942 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce6782-c7ac-4853-9f09-f9211a581ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:29:57.754974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce6782-c7ac-4853-9f09-f9211a581ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:29:57.758083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 639/1307] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:29:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:29:57.874778 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-17ce6782-c7ac-4853-9f09-f9211a581ad6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:29:57.876032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-17ce6782-c7ac-4853-9f09-f9211a581ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:29:58.006716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 669/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:29:58.310093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acd125-c6c8-4e7d-833e-e7b2dded8641 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:29:58.310371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acd125-c6c8-4e7d-833e-e7b2dded8641 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:29:58.310828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acd125-c6c8-4e7d-833e-e7b2dded8641 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:29:58.312198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 640/1309] 10.222.0.52 () {54 vars in 964 bytes} [Wed Sep 28 08:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:29:58.318999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33844edd-d290-4500-a4b8-b113147396d1 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:29:58.319298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33844edd-d290-4500-a4b8-b113147396d1 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:29:58.319568 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33844edd-d290-4500-a4b8-b113147396d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:29:58.418600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-33844edd-d290-4500-a4b8-b113147396d1 None None] MFA Rules not processed for user `736efafe9fc94b45bc75566b1696c498`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:29:58.419848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-33844edd-d290-4500-a4b8-b113147396d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:29:58.441785 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:29:58.503283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 670/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Sep 28 08:29:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:29:58.512677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acd125-c6c8-4e7d-833e-e7b2dded8641 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:29:58.515833 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-28acd125-c6c8-4e7d-833e-e7b2dded8641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:29:58.629944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acd125-c6c8-4e7d-833e-e7b2dded8641 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:29:58.711318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4c906-380f-4e43-a854-b9fb230192a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:29:58.711927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4c906-380f-4e43-a854-b9fb230192a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:29:58.712206 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4c906-380f-4e43-a854-b9fb230192a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:29:58.712476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4c906-380f-4e43-a854-b9fb230192a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:29:58.714638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ebc4c906-380f-4e43-a854-b9fb230192a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:29:58.774483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:29:58.834982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4c906-380f-4e43-a854-b9fb230192a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:29:58.836570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4c906-380f-4e43-a854-b9fb230192a7 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:29:58.858162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 641/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:30:01.340734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde57860-19b5-4561-999c-fb247424b077 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:01.341295 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde57860-19b5-4561-999c-fb247424b077 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:01.341295 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde57860-19b5-4561-999c-fb247424b077 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:01.342581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 671/1312] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:01.347895 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87ea6e-2f33-4bb0-b44a-bb53407a6612 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:01.348306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87ea6e-2f33-4bb0-b44a-bb53407a6612 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:01.348444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87ea6e-2f33-4bb0-b44a-bb53407a6612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:01.420985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ab87ea6e-2f33-4bb0-b44a-bb53407a6612 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:01.422240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ab87ea6e-2f33-4bb0-b44a-bb53407a6612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:01.521139 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 642/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:01.811072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde57860-19b5-4561-999c-fb247424b077 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:01.814123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-bde57860-19b5-4561-999c-fb247424b077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:01.935388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde57860-19b5-4561-999c-fb247424b077 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:02.020480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:02.023005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9f2d8-f827-43e1-b9bd-0d9fbe64cf13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:02.023703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9f2d8-f827-43e1-b9bd-0d9fbe64cf13 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:02.024187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9f2d8-f827-43e1-b9bd-0d9fbe64cf13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:02.024569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9f2d8-f827-43e1-b9bd-0d9fbe64cf13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:02.026742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-94b9f2d8-f827-43e1-b9bd-0d9fbe64cf13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:02.163785 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9f2d8-f827-43e1-b9bd-0d9fbe64cf13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:02.165167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9f2d8-f827-43e1-b9bd-0d9fbe64cf13 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:02.184663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 672/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:30:04.687554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfaf90-2a02-418f-9756-45276dd8e031 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:04.688104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfaf90-2a02-418f-9756-45276dd8e031 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:04.688104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfaf90-2a02-418f-9756-45276dd8e031 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:04.689480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 643/1315] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:04.694641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f831712-2764-4ee8-b222-f89f14d9d4d3 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:04.695064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f831712-2764-4ee8-b222-f89f14d9d4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:04.695195 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f831712-2764-4ee8-b222-f89f14d9d4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:04.761696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-9f831712-2764-4ee8-b222-f89f14d9d4d3 None None] MFA Rules not processed for user `0ff656a811aa4e698617e13e80ad9d73`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:04.762917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9f831712-2764-4ee8-b222-f89f14d9d4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:04.814756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:04.865755 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 673/1316] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:30:05.169592 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfaf90-2a02-418f-9756-45276dd8e031 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:05.171104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-06dfaf90-2a02-418f-9756-45276dd8e031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:05.184222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:05.277665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfaf90-2a02-418f-9756-45276dd8e031 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:05.351149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4c786-2779-4744-a90a-5cc36b8bccd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:05.351747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4c786-2779-4744-a90a-5cc36b8bccd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:05.352029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4c786-2779-4744-a90a-5cc36b8bccd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:05.352303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4c786-2779-4744-a90a-5cc36b8bccd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:05.353950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-87a4c786-2779-4744-a90a-5cc36b8bccd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:05.418244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:05.466424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4c786-2779-4744-a90a-5cc36b8bccd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:05.467867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4c786-2779-4744-a90a-5cc36b8bccd3 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:05.485619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 644/1317] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:05 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:30:08.201031 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317bfb7-61c6-49d5-8fbd-112858208347 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:08.201589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317bfb7-61c6-49d5-8fbd-112858208347 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:08.201713 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317bfb7-61c6-49d5-8fbd-112858208347 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:08.202960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 674/1318] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:08.208232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa556ae2-817f-4d45-8087-570800592667 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:08.208709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa556ae2-817f-4d45-8087-570800592667 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:08.208801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa556ae2-817f-4d45-8087-570800592667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:08.274743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-aa556ae2-817f-4d45-8087-570800592667 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:08.276493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-aa556ae2-817f-4d45-8087-570800592667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:08.372137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 645/1319] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:08.656059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317bfb7-61c6-49d5-8fbd-112858208347 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:08.657646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4317bfb7-61c6-49d5-8fbd-112858208347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:08.760345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317bfb7-61c6-49d5-8fbd-112858208347 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:08.834229 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f16b8-f871-480e-8487-a6a64983d2de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:08.834781 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f16b8-f871-480e-8487-a6a64983d2de service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:08.835058 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f16b8-f871-480e-8487-a6a64983d2de service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:08.835330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f16b8-f871-480e-8487-a6a64983d2de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:08.836920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-627f16b8-f871-480e-8487-a6a64983d2de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:08.844498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:08.950383 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f16b8-f871-480e-8487-a6a64983d2de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:08.951857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f16b8-f871-480e-8487-a6a64983d2de service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:08.970046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 675/1320] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:30:11.813181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156ad70-f421-41ae-afaf-89d0fee4f175 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:11.813689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156ad70-f421-41ae-afaf-89d0fee4f175 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:11.813779 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156ad70-f421-41ae-afaf-89d0fee4f175 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:11.815086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 646/1321] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:11.820641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dcded-75cf-4f66-90f4-bd2a3d516ea5 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:11.821165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dcded-75cf-4f66-90f4-bd2a3d516ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:11.821284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dcded-75cf-4f66-90f4-bd2a3d516ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:11.907843 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-cb7dcded-75cf-4f66-90f4-bd2a3d516ea5 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:11.909104 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-cb7dcded-75cf-4f66-90f4-bd2a3d516ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:11.982073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:12.033178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 676/1322] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:12.344646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c9b58-4567-427c-925a-858171cc7577 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:12.344930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c9b58-4567-427c-925a-858171cc7577 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:12.345273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c9b58-4567-427c-925a-858171cc7577 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:12.346929 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 647/1323] 10.222.0.52 () {54 vars in 964 bytes} [Wed Sep 28 08:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:12.352689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8b6e8-704c-41b9-93ee-0dd7ae4a4784 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:12.353798 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8b6e8-704c-41b9-93ee-0dd7ae4a4784 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:12.354259 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8b6e8-704c-41b9-93ee-0dd7ae4a4784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:12.443669 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-01b8b6e8-704c-41b9-93ee-0dd7ae4a4784 None None] MFA Rules not processed for user `736efafe9fc94b45bc75566b1696c498`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:12.445002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-01b8b6e8-704c-41b9-93ee-0dd7ae4a4784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:12.532859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 677/1324] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Sep 28 08:30:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:12.542596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c9b58-4567-427c-925a-858171cc7577 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:12.544119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a96c9b58-4567-427c-925a-858171cc7577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:12.565472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:12.659516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c9b58-4567-427c-925a-858171cc7577 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:12.740009 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a981f-0ff1-46b6-a521-4e958a523a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:12.740623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a981f-0ff1-46b6-a521-4e958a523a19 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:12.740904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a981f-0ff1-46b6-a521-4e958a523a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:12.741202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a981f-0ff1-46b6-a521-4e958a523a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:12.742832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-384a981f-0ff1-46b6-a521-4e958a523a19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:12.829048 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:12.872661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a981f-0ff1-46b6-a521-4e958a523a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:12.874174 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a981f-0ff1-46b6-a521-4e958a523a19 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:12.894311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 648/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:30:15.376102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848cab3-9cab-4ad4-9da2-1e480bb701c8 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:15.376674 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848cab3-9cab-4ad4-9da2-1e480bb701c8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:15.376674 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848cab3-9cab-4ad4-9da2-1e480bb701c8 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:15.377919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 678/1326] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:15.383445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d047a-7a83-48c9-bd56-787f93bb005d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:15.383906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d047a-7a83-48c9-bd56-787f93bb005d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:15.384114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d047a-7a83-48c9-bd56-787f93bb005d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:15.454001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-150d047a-7a83-48c9-bd56-787f93bb005d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:15.455215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-150d047a-7a83-48c9-bd56-787f93bb005d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:15.562883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 649/1327] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:15.844790 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848cab3-9cab-4ad4-9da2-1e480bb701c8 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:15.846364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3848cab3-9cab-4ad4-9da2-1e480bb701c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:15.907602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:15.953180 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848cab3-9cab-4ad4-9da2-1e480bb701c8 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:16.025119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b0060-da0d-43b9-8ce4-4a6b5349a61f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:16.025733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b0060-da0d-43b9-8ce4-4a6b5349a61f service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:16.026010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b0060-da0d-43b9-8ce4-4a6b5349a61f service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:16.026284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b0060-da0d-43b9-8ce4-4a6b5349a61f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:16.027853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-6a0b0060-da0d-43b9-8ce4-4a6b5349a61f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:16.138073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:16.141285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b0060-da0d-43b9-8ce4-4a6b5349a61f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:16.142844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b0060-da0d-43b9-8ce4-4a6b5349a61f service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:16.161890 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 679/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:30:18.438954 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30ef73-1ed4-4795-9a26-5eabee94d6e8 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:18.439468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30ef73-1ed4-4795-9a26-5eabee94d6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:18.439573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30ef73-1ed4-4795-9a26-5eabee94d6e8 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:18.440879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 650/1329] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:18.448243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeda758-d539-47f5-bfdb-4a9471991858 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:18.448911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeda758-d539-47f5-bfdb-4a9471991858 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:18.448911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeda758-d539-47f5-bfdb-4a9471991858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:18.520325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-1aeda758-d539-47f5-bfdb-4a9471991858 None None] MFA Rules not processed for user `0ff656a811aa4e698617e13e80ad9d73`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:18.521584 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1aeda758-d539-47f5-bfdb-4a9471991858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:18.624697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 680/1330] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:18.915327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30ef73-1ed4-4795-9a26-5eabee94d6e8 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:18.917299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2c30ef73-1ed4-4795-9a26-5eabee94d6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:18.948906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:19.027577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30ef73-1ed4-4795-9a26-5eabee94d6e8 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:19.135114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca33b4-4f8c-45df-957a-b98fc24ee8b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:19.135844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca33b4-4f8c-45df-957a-b98fc24ee8b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:19.136192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca33b4-4f8c-45df-957a-b98fc24ee8b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:19.136558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca33b4-4f8c-45df-957a-b98fc24ee8b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:19.138454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c6ca33b4-4f8c-45df-957a-b98fc24ee8b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:19.226905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:19.261513 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca33b4-4f8c-45df-957a-b98fc24ee8b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:19.262899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca33b4-4f8c-45df-957a-b98fc24ee8b7 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:19.282022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 651/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:30:21.736256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06a865-c916-43eb-ae3d-4e062095ade3 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:21.736809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06a865-c916-43eb-ae3d-4e062095ade3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:21.736809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06a865-c916-43eb-ae3d-4e062095ade3 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:21.738221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 681/1332] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:21.743522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496e0da-df1e-4d37-a6d3-7783e5b43f9a None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:21.743968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496e0da-df1e-4d37-a6d3-7783e5b43f9a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:21.744099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496e0da-df1e-4d37-a6d3-7783e5b43f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:21.814516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-d496e0da-df1e-4d37-a6d3-7783e5b43f9a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:21.815663 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d496e0da-df1e-4d37-a6d3-7783e5b43f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:21.917911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 652/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:22.205936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06a865-c916-43eb-ae3d-4e062095ade3 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:22.207890 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8d06a865-c916-43eb-ae3d-4e062095ade3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:22.279507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:22.315583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06a865-c916-43eb-ae3d-4e062095ade3 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:22.389786 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a110b8-839d-4d34-8eca-3306af1fd0e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:22.390392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a110b8-839d-4d34-8eca-3306af1fd0e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:22.390669 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a110b8-839d-4d34-8eca-3306af1fd0e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:22.390938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a110b8-839d-4d34-8eca-3306af1fd0e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:22.392542 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a7a110b8-839d-4d34-8eca-3306af1fd0e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:22.512658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a110b8-839d-4d34-8eca-3306af1fd0e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:22.514136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a110b8-839d-4d34-8eca-3306af1fd0e7 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:22.524326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:22.534405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 682/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:30:24.765109 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190a1b8-4bde-447b-a220-e76175daf4b0 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:24.765649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190a1b8-4bde-447b-a220-e76175daf4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:24.765649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190a1b8-4bde-447b-a220-e76175daf4b0 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:24.767019 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 653/1335] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:24.772552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9440f-f259-422e-bbde-02d32d674777 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:24.773000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9440f-f259-422e-bbde-02d32d674777 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:24.773170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9440f-f259-422e-bbde-02d32d674777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:24.842174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-21c9440f-f259-422e-bbde-02d32d674777 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:24.843304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-21c9440f-f259-422e-bbde-02d32d674777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:24.942759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 683/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:25.233231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190a1b8-4bde-447b-a220-e76175daf4b0 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:25.234759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6190a1b8-4bde-447b-a220-e76175daf4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:25.277099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:25.342566 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190a1b8-4bde-447b-a220-e76175daf4b0 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:25.418817 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6ab2c-67be-45e0-9cf8-bf644e99edcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:25.419426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6ab2c-67be-45e0-9cf8-bf644e99edcb service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:25.419710 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6ab2c-67be-45e0-9cf8-bf644e99edcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:25.419982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6ab2c-67be-45e0-9cf8-bf644e99edcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:25.421602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6fa6ab2c-67be-45e0-9cf8-bf644e99edcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:25.506025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:25.532837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6ab2c-67be-45e0-9cf8-bf644e99edcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:25.534310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6ab2c-67be-45e0-9cf8-bf644e99edcb service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:25.553036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 654/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:30:28.065717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71997d-172e-4e72-ac16-a0837f3a6988 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:28.066237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71997d-172e-4e72-ac16-a0837f3a6988 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:28.066237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71997d-172e-4e72-ac16-a0837f3a6988 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:28.067519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 684/1338] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:28.072831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5cadb-d38a-44ad-8757-4646a13a4b1d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:28.073260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5cadb-d38a-44ad-8757-4646a13a4b1d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:28.073406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5cadb-d38a-44ad-8757-4646a13a4b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:28.140151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-d5b5cadb-d38a-44ad-8757-4646a13a4b1d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:28.141392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d5b5cadb-d38a-44ad-8757-4646a13a4b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:28.240985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 655/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:28.540541 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71997d-172e-4e72-ac16-a0837f3a6988 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:28.542196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ea71997d-172e-4e72-ac16-a0837f3a6988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:28.623276 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:28.653377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71997d-172e-4e72-ac16-a0837f3a6988 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:28.731594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4945cc-dfac-47eb-8903-69ce0800c53b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:28.732575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4945cc-dfac-47eb-8903-69ce0800c53b service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:28.733080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4945cc-dfac-47eb-8903-69ce0800c53b service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:28.733627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4945cc-dfac-47eb-8903-69ce0800c53b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:28.736067 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-cb4945cc-dfac-47eb-8903-69ce0800c53b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:28.861815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4945cc-dfac-47eb-8903-69ce0800c53b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:28.863160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4945cc-dfac-47eb-8903-69ce0800c53b service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:28.881902 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 685/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:30:33.141041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510de54-63c3-4519-92c4-e6210170a2f9 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:33.141532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510de54-63c3-4519-92c4-e6210170a2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:33.141633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510de54-63c3-4519-92c4-e6210170a2f9 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:33.142956 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 656/1341] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:33.149209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce16af-da99-4fe2-9244-fc1aa337819d None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:33.150256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce16af-da99-4fe2-9244-fc1aa337819d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:33.150406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce16af-da99-4fe2-9244-fc1aa337819d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:33.156083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:33.244662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-10ce16af-da99-4fe2-9244-fc1aa337819d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:33.247405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-10ce16af-da99-4fe2-9244-fc1aa337819d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:33.380645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 686/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:33.698225 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510de54-63c3-4519-92c4-e6210170a2f9 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:33.699863 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8510de54-63c3-4519-92c4-e6210170a2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:33.760905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:33.823797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510de54-63c3-4519-92c4-e6210170a2f9 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:33.909128 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7bb80-6bdb-4ac8-9948-d0a318f372c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:33.909858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7bb80-6bdb-4ac8-9948-d0a318f372c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:33.910196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7bb80-6bdb-4ac8-9948-d0a318f372c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:33.910450 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7bb80-6bdb-4ac8-9948-d0a318f372c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:33.912065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b4e7bb80-6bdb-4ac8-9948-d0a318f372c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:34.021044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:34.039109 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7bb80-6bdb-4ac8-9948-d0a318f372c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:34.041180 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7bb80-6bdb-4ac8-9948-d0a318f372c1 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:34.061378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 657/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:30:37.030725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83f9f1-ff6b-4658-ba5a-bcaa25b079b2 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:37.031518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83f9f1-ff6b-4658-ba5a-bcaa25b079b2 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:37.031518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83f9f1-ff6b-4658-ba5a-bcaa25b079b2 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:37.032876 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 687/1344] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:37.039210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7209cd-53a8-43e5-976d-111f97974099 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:37.039678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7209cd-53a8-43e5-976d-111f97974099 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:37.039759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7209cd-53a8-43e5-976d-111f97974099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:37.109472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-5f7209cd-53a8-43e5-976d-111f97974099 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:37.110672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5f7209cd-53a8-43e5-976d-111f97974099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:37.209406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 658/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:37.488884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83f9f1-ff6b-4658-ba5a-bcaa25b079b2 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:37.490503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-eb83f9f1-ff6b-4658-ba5a-bcaa25b079b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:37.589642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:37.609757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83f9f1-ff6b-4658-ba5a-bcaa25b079b2 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:37.689057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63adac6-b426-4dc9-8aa6-146604b47c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:37.689690 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63adac6-b426-4dc9-8aa6-146604b47c9b service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:37.689994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63adac6-b426-4dc9-8aa6-146604b47c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:37.690267 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63adac6-b426-4dc9-8aa6-146604b47c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:37.691856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e63adac6-b426-4dc9-8aa6-146604b47c9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:37.812387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63adac6-b426-4dc9-8aa6-146604b47c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:37.814277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63adac6-b426-4dc9-8aa6-146604b47c9b service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:37.836705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 688/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:30:41.999327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c47c2-de07-4dc0-ac21-07bbf107cba1 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:41.999827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c47c2-de07-4dc0-ac21-07bbf107cba1 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:41.999921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c47c2-de07-4dc0-ac21-07bbf107cba1 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:42.001276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 659/1347] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:42.007261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5be6f-c853-4498-8079-aa1991cb44c2 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:42.007768 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5be6f-c853-4498-8079-aa1991cb44c2 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:42.007861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5be6f-c853-4498-8079-aa1991cb44c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:42.025779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:42.082194 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2ee5be6f-c853-4498-8079-aa1991cb44c2 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:42.083441 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2ee5be6f-c853-4498-8079-aa1991cb44c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:42.186646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 689/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:42.468051 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c47c2-de07-4dc0-ac21-07bbf107cba1 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:42.469615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f84c47c2-de07-4dc0-ac21-07bbf107cba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:42.529350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:42.578498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c47c2-de07-4dc0-ac21-07bbf107cba1 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:42.656174 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6fe90-c96b-4169-b9bb-feed77bbdcdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:42.657080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6fe90-c96b-4169-b9bb-feed77bbdcdc service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:42.657384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6fe90-c96b-4169-b9bb-feed77bbdcdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:42.657717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6fe90-c96b-4169-b9bb-feed77bbdcdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:42.659502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-fdc6fe90-c96b-4169-b9bb-feed77bbdcdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:42.767419 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:42.771712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6fe90-c96b-4169-b9bb-feed77bbdcdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:42.774004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6fe90-c96b-4169-b9bb-feed77bbdcdc service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:42.794446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 660/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:30:47.817754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed27662-5803-4802-bff5-0162c51220b5 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:47.818220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed27662-5803-4802-bff5-0162c51220b5 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:47.818319 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed27662-5803-4802-bff5-0162c51220b5 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:47.819915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 690/1350] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:47.825826 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf4b89-a3fc-4816-afec-ccf4f668f09d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:47.826289 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf4b89-a3fc-4816-afec-ccf4f668f09d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:47.826388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf4b89-a3fc-4816-afec-ccf4f668f09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:47.917965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-4caf4b89-a3fc-4816-afec-ccf4f668f09d None None] MFA Rules not processed for user `6b40ee263df74e58bf3f958c01eb742f`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:47.919266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4caf4b89-a3fc-4816-afec-ccf4f668f09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:48.107625 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 661/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:30:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:30:52.185047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=59 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:52.376122 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebc908-8378-446e-b15c-a01365c0121f None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:52.376656 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebc908-8378-446e-b15c-a01365c0121f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:52.377056 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebc908-8378-446e-b15c-a01365c0121f None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:52.378606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 691/1352] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:30:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:52.384001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efa466-81ae-408b-b730-29e0ce26212c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:52.384465 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efa466-81ae-408b-b730-29e0ce26212c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:52.384549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efa466-81ae-408b-b730-29e0ce26212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:52.444013 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:52.471513 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-d4efa466-81ae-408b-b730-29e0ce26212c None None] MFA Rules not processed for user `6b40ee263df74e58bf3f958c01eb742f`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:52.472745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d4efa466-81ae-408b-b730-29e0ce26212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:52.598570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 662/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:30:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:54.004625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147e7b6-1568-4b12-b851-e59702e69b72 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:54.005166 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147e7b6-1568-4b12-b851-e59702e69b72 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:54.005258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147e7b6-1568-4b12-b851-e59702e69b72 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:54.006558 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 692/1354] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:54.011608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6ea04-0464-42aa-a520-725dbba7717e None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:54.012020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6ea04-0464-42aa-a520-725dbba7717e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:54.012182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6ea04-0464-42aa-a520-725dbba7717e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:54.088740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-3eb6ea04-0464-42aa-a520-725dbba7717e None None] MFA Rules not processed for user `6b40ee263df74e58bf3f958c01eb742f`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:54.089964 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3eb6ea04-0464-42aa-a520-725dbba7717e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:54.132029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:54.199774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 663/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:30:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:57.827890 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b736609-aacf-4c5c-afec-0a77dd8b5ca4 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:57.828922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b736609-aacf-4c5c-afec-0a77dd8b5ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:57.830698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b736609-aacf-4c5c-afec-0a77dd8b5ca4 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:57.833192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 693/1356] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:30:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:57.839119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ff5f7-7780-4ece-ac42-f3a3705dec88 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:57.839604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ff5f7-7780-4ece-ac42-f3a3705dec88 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:57.839704 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ff5f7-7780-4ece-ac42-f3a3705dec88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:57.917994 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-b28ff5f7-7780-4ece-ac42-f3a3705dec88 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:57.919382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b28ff5f7-7780-4ece-ac42-f3a3705dec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:58.029257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 664/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:30:58.337964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c88ef-2d11-49c9-8f9e-c9c6e57a71c8 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:58.338253 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c88ef-2d11-49c9-8f9e-c9c6e57a71c8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:58.338758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c88ef-2d11-49c9-8f9e-c9c6e57a71c8 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:58.340291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 694/1358] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 08:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:30:58.348895 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe3227-892c-4190-829b-81363b0dd4d2 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:58.349429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe3227-892c-4190-829b-81363b0dd4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:58.349859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe3227-892c-4190-829b-81363b0dd4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:58.380561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:58.452043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-2fbe3227-892c-4190-829b-81363b0dd4d2 None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:30:58.453401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2fbe3227-892c-4190-829b-81363b0dd4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:58.591432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 665/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 08:30:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:30:58.599466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c88ef-2d11-49c9-8f9e-c9c6e57a71c8 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:30:58.600974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f20c88ef-2d11-49c9-8f9e-c9c6e57a71c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:58.758775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c88ef-2d11-49c9-8f9e-c9c6e57a71c8 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:30:58.842414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:30:58.887508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e4a15-5972-41cb-8c2c-4de6b3b6df60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:30:58.888373 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e4a15-5972-41cb-8c2c-4de6b3b6df60 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:30:58.888752 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e4a15-5972-41cb-8c2c-4de6b3b6df60 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:30:58.889176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e4a15-5972-41cb-8c2c-4de6b3b6df60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:30:58.890876 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1b7e4a15-5972-41cb-8c2c-4de6b3b6df60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:30:58.996952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e4a15-5972-41cb-8c2c-4de6b3b6df60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:30:58.998606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e4a15-5972-41cb-8c2c-4de6b3b6df60 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:30:59.018526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 695/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:30:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:01.434099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9d073-1b37-4981-9f2c-bdc6958f3203 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:01.434612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9d073-1b37-4981-9f2c-bdc6958f3203 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:01.434612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9d073-1b37-4981-9f2c-bdc6958f3203 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:01.437281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 666/1361] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:01.441561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d65914-5c6b-48bb-8c6c-1bda547fa8db None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:01.442177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d65914-5c6b-48bb-8c6c-1bda547fa8db None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:01.442177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d65914-5c6b-48bb-8c6c-1bda547fa8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:01.510784 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-66d65914-5c6b-48bb-8c6c-1bda547fa8db None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:01.511973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-66d65914-5c6b-48bb-8c6c-1bda547fa8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:01.556232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:01.627815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 696/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:01.923013 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9d073-1b37-4981-9f2c-bdc6958f3203 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:01.924527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-dea9d073-1b37-4981-9f2c-bdc6958f3203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:01.997364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:02.116764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9d073-1b37-4981-9f2c-bdc6958f3203 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:02.267844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03564dbd-e44c-42f0-80c3-c07ae03113d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:02.268453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564dbd-e44c-42f0-80c3-c07ae03113d7 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:02.268729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564dbd-e44c-42f0-80c3-c07ae03113d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:02.268993 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564dbd-e44c-42f0-80c3-c07ae03113d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:02.270624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-03564dbd-e44c-42f0-80c3-c07ae03113d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:02.295673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:02.368219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03564dbd-e44c-42f0-80c3-c07ae03113d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:02.369868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03564dbd-e44c-42f0-80c3-c07ae03113d7 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:02.389162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 667/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:04.926689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917359cc-33f1-45d1-8930-608b82810e51 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:04.927244 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917359cc-33f1-45d1-8930-608b82810e51 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:04.927244 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917359cc-33f1-45d1-8930-608b82810e51 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:04.928793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 697/1364] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:04.933937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e79f42-7b35-4475-9c6a-0177603e0d8d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:04.934342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e79f42-7b35-4475-9c6a-0177603e0d8d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:04.934507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e79f42-7b35-4475-9c6a-0177603e0d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:05.003869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-e8e79f42-7b35-4475-9c6a-0177603e0d8d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:05.005103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e8e79f42-7b35-4475-9c6a-0177603e0d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:05.112544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:05.123478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 668/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:05.412108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f093a5-c16f-4fe3-b616-ce13819a065e None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:05.412456 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f093a5-c16f-4fe3-b616-ce13819a065e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:05.412804 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f093a5-c16f-4fe3-b616-ce13819a065e None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:05.414223 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 698/1366] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 08:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:05.420373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d2979-2cb8-4e25-b9fb-1db64b578e87 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:05.420594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d2979-2cb8-4e25-b9fb-1db64b578e87 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:05.420943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d2979-2cb8-4e25-b9fb-1db64b578e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:05.508039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ec2d2979-2cb8-4e25-b9fb-1db64b578e87 None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:05.509897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ec2d2979-2cb8-4e25-b9fb-1db64b578e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:05.645691 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 669/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 08:31:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:31:05.653976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f093a5-c16f-4fe3-b616-ce13819a065e None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:05.655923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a4f093a5-c16f-4fe3-b616-ce13819a065e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:05.821048 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f093a5-c16f-4fe3-b616-ce13819a065e None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:05.842815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:05.945835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265c105-043b-4100-8d59-7936f3245056 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:05.946601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265c105-043b-4100-8d59-7936f3245056 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:05.946809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265c105-043b-4100-8d59-7936f3245056 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:05.947066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265c105-043b-4100-8d59-7936f3245056 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:05.948733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d265c105-043b-4100-8d59-7936f3245056 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:06.049212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265c105-043b-4100-8d59-7936f3245056 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:06.050900 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265c105-043b-4100-8d59-7936f3245056 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:06.073980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 699/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:06.172974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8bd1-2dea-4c82-a266-d76f49c41183 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:06.173563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8bd1-2dea-4c82-a266-d76f49c41183 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:06.173657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8bd1-2dea-4c82-a266-d76f49c41183 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:06.175040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 670/1369] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:06.181270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1790eb-e8b8-496d-9166-80cf2c5e657f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:06.181557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1790eb-e8b8-496d-9166-80cf2c5e657f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:06.181831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1790eb-e8b8-496d-9166-80cf2c5e657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:06.220323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:06.264871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-9e1790eb-e8b8-496d-9166-80cf2c5e657f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:06.267747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9e1790eb-e8b8-496d-9166-80cf2c5e657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:06.389126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 700/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:06.689633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2ed57-79e0-4aca-9e79-19575d8e39a6 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:06.690077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2ed57-79e0-4aca-9e79-19575d8e39a6 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:06.690454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2ed57-79e0-4aca-9e79-19575d8e39a6 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:06.691956 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 671/1371] 10.222.0.52 () {54 vars in 963 bytes} [Wed Sep 28 08:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:31:06.697226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfdc2b-9a6d-441f-a2a0-c0985899bfd2 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:06.697516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfdc2b-9a6d-441f-a2a0-c0985899bfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:06.697791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfdc2b-9a6d-441f-a2a0-c0985899bfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:06.781247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-7bcfdc2b-9a6d-441f-a2a0-c0985899bfd2 None None] MFA Rules not processed for user `5274740e5d7c4ab1ba9e9781815532c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:06.782631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7bcfdc2b-9a6d-441f-a2a0-c0985899bfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:06.835060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:06.871544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 701/1372] 10.222.0.52 () {58 vars in 1060 bytes} [Wed Sep 28 08:31:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:06.879049 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2ed57-79e0-4aca-9e79-19575d8e39a6 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:06.880817 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9ca2ed57-79e0-4aca-9e79-19575d8e39a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:06.938814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:06.999288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2ed57-79e0-4aca-9e79-19575d8e39a6 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:07.083693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27514f-b4e9-41e9-b8a3-d93c9c7dc663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:07.083693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27514f-b4e9-41e9-b8a3-d93c9c7dc663 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:07.083693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27514f-b4e9-41e9-b8a3-d93c9c7dc663 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:07.084045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27514f-b4e9-41e9-b8a3-d93c9c7dc663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:07.085490 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4b27514f-b4e9-41e9-b8a3-d93c9c7dc663 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:07.207607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:07.226213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27514f-b4e9-41e9-b8a3-d93c9c7dc663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:07.228665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27514f-b4e9-41e9-b8a3-d93c9c7dc663 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:07.258156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 672/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:31:08.370576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b3edd-95e6-4263-9097-1197b7efad5d None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:08.371133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b3edd-95e6-4263-9097-1197b7efad5d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:08.371133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b3edd-95e6-4263-9097-1197b7efad5d None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:08.372570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 702/1374] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:08.377974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7723974-fa47-44e3-9089-9ad9ccaeaca6 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:08.378392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7723974-fa47-44e3-9089-9ad9ccaeaca6 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:08.378539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7723974-fa47-44e3-9089-9ad9ccaeaca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:08.447125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-a7723974-fa47-44e3-9089-9ad9ccaeaca6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:08.448353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a7723974-fa47-44e3-9089-9ad9ccaeaca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:08.552881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 673/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:08.844549 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b3edd-95e6-4263-9097-1197b7efad5d None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:08.846171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e11b3edd-95e6-4263-9097-1197b7efad5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:09.007316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b3edd-95e6-4263-9097-1197b7efad5d None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:09.061253 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:09.127552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d4c12-93fd-49d4-828b-cb72cd54775f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:09.128174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d4c12-93fd-49d4-828b-cb72cd54775f service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:09.128455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d4c12-93fd-49d4-828b-cb72cd54775f service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:09.128726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d4c12-93fd-49d4-828b-cb72cd54775f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:09.130364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a52d4c12-93fd-49d4-828b-cb72cd54775f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:09.233913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d4c12-93fd-49d4-828b-cb72cd54775f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:09.235972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d4c12-93fd-49d4-828b-cb72cd54775f service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:09.255636 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 703/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:11.500815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe2328-0ad3-4e6a-8042-6fe96aaeb941 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:11.501421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe2328-0ad3-4e6a-8042-6fe96aaeb941 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:11.501818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe2328-0ad3-4e6a-8042-6fe96aaeb941 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:11.503423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 674/1377] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:11.509407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df69f5b-6355-470e-afc6-71372bab6c9e None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:11.509848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df69f5b-6355-470e-afc6-71372bab6c9e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:11.510030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df69f5b-6355-470e-afc6-71372bab6c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:11.587409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-0df69f5b-6355-470e-afc6-71372bab6c9e None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:11.588646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-0df69f5b-6355-470e-afc6-71372bab6c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:11.597221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:11.709177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 704/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:12.002525 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe2328-0ad3-4e6a-8042-6fe96aaeb941 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:12.004315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5abe2328-0ad3-4e6a-8042-6fe96aaeb941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:12.074090 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:12.178677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe2328-0ad3-4e6a-8042-6fe96aaeb941 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:12.310498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12daa46-83d7-4dca-bdb7-77bc15e76616 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:12.311012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12daa46-83d7-4dca-bdb7-77bc15e76616 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:12.311284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12daa46-83d7-4dca-bdb7-77bc15e76616 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:12.311547 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12daa46-83d7-4dca-bdb7-77bc15e76616 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:12.313150 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d12daa46-83d7-4dca-bdb7-77bc15e76616 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:12.333233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:12.409075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12daa46-83d7-4dca-bdb7-77bc15e76616 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:12.410677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12daa46-83d7-4dca-bdb7-77bc15e76616 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:12.429261 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 675/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:14.662839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc38e3-1bbb-432f-87d1-fa115ae906df None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:14.663418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc38e3-1bbb-432f-87d1-fa115ae906df None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:14.663418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc38e3-1bbb-432f-87d1-fa115ae906df None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:14.665875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 705/1380] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:31:14.671284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d310efc-dd93-4dfc-ba3d-369c12048ea4 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:14.671844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d310efc-dd93-4dfc-ba3d-369c12048ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:14.671844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d310efc-dd93-4dfc-ba3d-369c12048ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:14.751863 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-9d310efc-dd93-4dfc-ba3d-369c12048ea4 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:14.753067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9d310efc-dd93-4dfc-ba3d-369c12048ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:14.876362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:14.894918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 676/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:15.183712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc38e3-1bbb-432f-87d1-fa115ae906df None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:15.185747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-adfc38e3-1bbb-432f-87d1-fa115ae906df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:15.331518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:15.343664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc38e3-1bbb-432f-87d1-fa115ae906df None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:15.479447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845175b-c82e-4f39-ae87-1a6c5f3fe4cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:15.480057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845175b-c82e-4f39-ae87-1a6c5f3fe4cf service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:15.480332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845175b-c82e-4f39-ae87-1a6c5f3fe4cf service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:15.480600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845175b-c82e-4f39-ae87-1a6c5f3fe4cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:15.482259 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5845175b-c82e-4f39-ae87-1a6c5f3fe4cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:15.584312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5845175b-c82e-4f39-ae87-1a6c5f3fe4cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:15.585981 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5845175b-c82e-4f39-ae87-1a6c5f3fe4cf service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:15.597468 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:15.608409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 706/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:18.006643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc10ca7-5f68-440a-8662-0be05c3512e0 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:18.007304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc10ca7-5f68-440a-8662-0be05c3512e0 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:18.007304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc10ca7-5f68-440a-8662-0be05c3512e0 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:18.009051 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 677/1383] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:18.014523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7925a24-7f82-4b0c-a6ce-a1a72130bd06 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:18.015162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7925a24-7f82-4b0c-a6ce-a1a72130bd06 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:18.015162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7925a24-7f82-4b0c-a6ce-a1a72130bd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:18.092063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f7925a24-7f82-4b0c-a6ce-a1a72130bd06 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:18.093348 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f7925a24-7f82-4b0c-a6ce-a1a72130bd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:18.214306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 707/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:18.604504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc10ca7-5f68-440a-8662-0be05c3512e0 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:18.606055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3dc10ca7-5f68-440a-8662-0be05c3512e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:18.778112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc10ca7-5f68-440a-8662-0be05c3512e0 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:18.862897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:18.910798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7905646-dc98-4288-a3ca-db6597eeca2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:18.911400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7905646-dc98-4288-a3ca-db6597eeca2d service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:18.911673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7905646-dc98-4288-a3ca-db6597eeca2d service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:18.911941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7905646-dc98-4288-a3ca-db6597eeca2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:18.913544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a7905646-dc98-4288-a3ca-db6597eeca2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:19.021314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7905646-dc98-4288-a3ca-db6597eeca2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:19.022916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7905646-dc98-4288-a3ca-db6597eeca2d service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:19.041315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 678/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:21.248330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f28984-6d8d-41c1-962f-d6f07098602f None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:21.248939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f28984-6d8d-41c1-962f-d6f07098602f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:21.248939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f28984-6d8d-41c1-962f-d6f07098602f None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:21.250321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 708/1386] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:21.255804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324685ad-f5a4-49cb-b63d-79b1cb46c23e None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:21.256270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324685ad-f5a4-49cb-b63d-79b1cb46c23e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:21.256387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324685ad-f5a4-49cb-b63d-79b1cb46c23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:21.325032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-324685ad-f5a4-49cb-b63d-79b1cb46c23e None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:21.326258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-324685ad-f5a4-49cb-b63d-79b1cb46c23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:21.360271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:21.425159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 679/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:21.731867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f28984-6d8d-41c1-962f-d6f07098602f None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:21.733397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-79f28984-6d8d-41c1-962f-d6f07098602f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:21.832964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:21.910221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f28984-6d8d-41c1-962f-d6f07098602f None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:22.023256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1953e-df3e-4fd1-b536-81db14e651b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:22.024314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1953e-df3e-4fd1-b536-81db14e651b9 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:22.024590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1953e-df3e-4fd1-b536-81db14e651b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:22.024864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1953e-df3e-4fd1-b536-81db14e651b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:22.026481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7ce1953e-df3e-4fd1-b536-81db14e651b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:22.066214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:22.122091 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1953e-df3e-4fd1-b536-81db14e651b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:22.123681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1953e-df3e-4fd1-b536-81db14e651b9 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:22.142502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 709/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:22.789674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8758af1-6495-4948-ae2e-5e1094dea375 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:22.790235 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8758af1-6495-4948-ae2e-5e1094dea375 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:22.790235 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8758af1-6495-4948-ae2e-5e1094dea375 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:22.791462 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 680/1389] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:22.797233 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcd4e4-74e4-4253-a97d-5a32c210859d None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:22.797679 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcd4e4-74e4-4253-a97d-5a32c210859d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:22.797907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dcd4e4-74e4-4253-a97d-5a32c210859d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:22.875602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-d7dcd4e4-74e4-4253-a97d-5a32c210859d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:22.876832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d7dcd4e4-74e4-4253-a97d-5a32c210859d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:22.985847 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 710/1390] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:23.297023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebf14a-f408-4fca-8abc-30e833059439 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:23.297340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebf14a-f408-4fca-8abc-30e833059439 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:23.297611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebf14a-f408-4fca-8abc-30e833059439 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:23.298940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 681/1391] 10.222.0.52 () {54 vars in 962 bytes} [Wed Sep 28 08:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:23.304160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867b754-9fd6-4b93-a9c6-fcdcdf0f57f9 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:23.304440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867b754-9fd6-4b93-a9c6-fcdcdf0f57f9 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:23.304711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867b754-9fd6-4b93-a9c6-fcdcdf0f57f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:23.321446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:23.384427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-4867b754-9fd6-4b93-a9c6-fcdcdf0f57f9 None None] MFA Rules not processed for user `ff22c2632c614fca8008feab93c8fa2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:23.385571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4867b754-9fd6-4b93-a9c6-fcdcdf0f57f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:23.465227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 711/1392] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Sep 28 08:31:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:23.474012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebf14a-f408-4fca-8abc-30e833059439 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:23.475473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5aebf14a-f408-4fca-8abc-30e833059439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:23.597560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebf14a-f408-4fca-8abc-30e833059439 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:23.662180 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:23.678443 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25f8ce-a580-4c4d-b9b5-8e2547805c8f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff22c2632c614fca8008feab93c8fa2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:23.679043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25f8ce-a580-4c4d-b9b5-8e2547805c8f service glance] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:23.679317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25f8ce-a580-4c4d-b9b5-8e2547805c8f service glance] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:23.679584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25f8ce-a580-4c4d-b9b5-8e2547805c8f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:23.681722 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9f25f8ce-a580-4c4d-b9b5-8e2547805c8f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:23.791546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25f8ce-a580-4c4d-b9b5-8e2547805c8f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:23.793104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25f8ce-a580-4c4d-b9b5-8e2547805c8f service glance] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:23.812126 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 682/1393] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:24.410783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b714b-0bb3-4d34-acc7-7f0a1886ebc6 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:24.411376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b714b-0bb3-4d34-acc7-7f0a1886ebc6 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:24.411376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b714b-0bb3-4d34-acc7-7f0a1886ebc6 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:24.412737 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 712/1394] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:24.418907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261528a-30af-4b02-9aa1-a450482bc2b0 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:24.419223 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261528a-30af-4b02-9aa1-a450482bc2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:24.419502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8261528a-30af-4b02-9aa1-a450482bc2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:24.494737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-8261528a-30af-4b02-9aa1-a450482bc2b0 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:24.496037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8261528a-30af-4b02-9aa1-a450482bc2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:24.543652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:24.574372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f109ebe-de40-4ff8-b85e-0d7d20a6792f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:24.574638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f109ebe-de40-4ff8-b85e-0d7d20a6792f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:24.574932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f109ebe-de40-4ff8-b85e-0d7d20a6792f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:24.611446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 683/1395] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:31:24.645849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-8f109ebe-de40-4ff8-b85e-0d7d20a6792f None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:24.647056 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8f109ebe-de40-4ff8-b85e-0d7d20a6792f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:24.705713 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:24.740097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 713/1396] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:31:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:31:24.783178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3836a39-bfba-4042-871b-22f7d4fbdb5f None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:24.783543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3836a39-bfba-4042-871b-22f7d4fbdb5f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:24.783883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3836a39-bfba-4042-871b-22f7d4fbdb5f None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:24.785385 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 684/1397] 10.222.0.52 () {54 vars in 943 bytes} [Wed Sep 28 08:31:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:31:24.792279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fe0dd-32ef-4ad0-a6fe-9a35cf9c7ac9 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:24.792715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fe0dd-32ef-4ad0-a6fe-9a35cf9c7ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:24.793063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fe0dd-32ef-4ad0-a6fe-9a35cf9c7ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:24.872005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-847fe0dd-32ef-4ad0-a6fe-9a35cf9c7ac9 None None] MFA Rules not processed for user `2f410e0148c1403098b9b73735425726`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:24.873346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-847fe0dd-32ef-4ad0-a6fe-9a35cf9c7ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:24.908495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3836a39-bfba-4042-871b-22f7d4fbdb5f None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:24.910054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a3836a39-bfba-4042-871b-22f7d4fbdb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:25.000793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 714/1398] 10.222.0.52 () {58 vars in 1040 bytes} [Wed Sep 28 08:31:24 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:31:25.010052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fe0dd-32ef-4ad0-a6fe-9a35cf9c7ac9 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:25.019512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fe0dd-32ef-4ad0-a6fe-9a35cf9c7ac9 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:25.047096 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:25.081877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3836a39-bfba-4042-871b-22f7d4fbdb5f None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:25.109145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:25.133283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a13fe-1c82-46ac-8bf5-511b79fd0b38 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:25.133888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a13fe-1c82-46ac-8bf5-511b79fd0b38 service swift] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:25.134163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a13fe-1c82-46ac-8bf5-511b79fd0b38 service swift] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:25.134428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a13fe-1c82-46ac-8bf5-511b79fd0b38 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:25.136042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-742a13fe-1c82-46ac-8bf5-511b79fd0b38 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:25.200943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32c9f5-d9a2-4886-a600-fb791049c6a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:25.201603 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32c9f5-d9a2-4886-a600-fb791049c6a2 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:25.201884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32c9f5-d9a2-4886-a600-fb791049c6a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:25.202160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32c9f5-d9a2-4886-a600-fb791049c6a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:25.203794 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-0e32c9f5-d9a2-4886-a600-fb791049c6a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:25.223564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a13fe-1c82-46ac-8bf5-511b79fd0b38 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:25.225253 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a13fe-1c82-46ac-8bf5-511b79fd0b38 service swift] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:25.233207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 715/1399] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:31:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 08:31:25.300704 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32c9f5-d9a2-4886-a600-fb791049c6a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:25.302445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32c9f5-d9a2-4886-a600-fb791049c6a2 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:25.323848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 685/1400] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:27.571688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed1fcd-417b-4b49-8b5e-9961347c9ff3 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:27.572666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed1fcd-417b-4b49-8b5e-9961347c9ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:27.572666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed1fcd-417b-4b49-8b5e-9961347c9ff3 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:27.573840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 716/1401] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:27.579653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7421700-5d87-462a-84be-46ddc61bec67 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:27.580739 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7421700-5d87-462a-84be-46ddc61bec67 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:27.581242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7421700-5d87-462a-84be-46ddc61bec67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:27.632624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:27.671106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-f7421700-5d87-462a-84be-46ddc61bec67 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:27.672318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f7421700-5d87-462a-84be-46ddc61bec67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:27.798742 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 686/1402] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:28.095220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed1fcd-417b-4b49-8b5e-9961347c9ff3 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:28.095581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-30ed1fcd-417b-4b49-8b5e-9961347c9ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:28.168734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:28.276088 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed1fcd-417b-4b49-8b5e-9961347c9ff3 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:28.413646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31259399-c6d2-47e3-a5a4-f8fdb7140f6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:28.414337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31259399-c6d2-47e3-a5a4-f8fdb7140f6a service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:28.414699 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31259399-c6d2-47e3-a5a4-f8fdb7140f6a service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:28.415040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31259399-c6d2-47e3-a5a4-f8fdb7140f6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:28.416726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-31259399-c6d2-47e3-a5a4-f8fdb7140f6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:28.438293 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:28.534567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31259399-c6d2-47e3-a5a4-f8fdb7140f6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:28.537479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31259399-c6d2-47e3-a5a4-f8fdb7140f6a service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:28.569304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 717/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:30.871166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f7f1f-2d8b-4299-ae9e-add2fe6173a3 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:30.871729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f7f1f-2d8b-4299-ae9e-add2fe6173a3 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:30.871729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f7f1f-2d8b-4299-ae9e-add2fe6173a3 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:30.873164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 687/1404] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:30.878739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d1351-4e93-4a10-94b1-4277436f2223 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:30.879262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d1351-4e93-4a10-94b1-4277436f2223 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:30.879371 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d1351-4e93-4a10-94b1-4277436f2223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:30.949976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f58d1351-4e93-4a10-94b1-4277436f2223 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:30.951612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f58d1351-4e93-4a10-94b1-4277436f2223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:31.060625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:31.067954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 718/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:31.393736 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f7f1f-2d8b-4299-ae9e-add2fe6173a3 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:31.395269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-983f7f1f-2d8b-4299-ae9e-add2fe6173a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:31.528672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:31.575542 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f7f1f-2d8b-4299-ae9e-add2fe6173a3 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:31.710818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f956e33-7243-4cb9-aa05-553e156a4af2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:31.712027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f956e33-7243-4cb9-aa05-553e156a4af2 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:31.712329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f956e33-7243-4cb9-aa05-553e156a4af2 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:31.712596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f956e33-7243-4cb9-aa05-553e156a4af2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:31.714209 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9f956e33-7243-4cb9-aa05-553e156a4af2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:31.801778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:31.828486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f956e33-7243-4cb9-aa05-553e156a4af2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:31.830241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f956e33-7243-4cb9-aa05-553e156a4af2 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:31.850598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 688/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:34.090972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79269e-6fbd-4c0d-b8a8-aed8b2ad1113 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:34.091547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79269e-6fbd-4c0d-b8a8-aed8b2ad1113 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:34.091547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79269e-6fbd-4c0d-b8a8-aed8b2ad1113 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:34.093205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 719/1407] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:34.098649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b9a84-3e6e-4719-bfa7-5b564866c020 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:34.099123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b9a84-3e6e-4719-bfa7-5b564866c020 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:34.099313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b9a84-3e6e-4719-bfa7-5b564866c020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:34.191623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-de0b9a84-3e6e-4719-bfa7-5b564866c020 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:34.193884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-de0b9a84-3e6e-4719-bfa7-5b564866c020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:34.341259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 689/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:34.631869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79269e-6fbd-4c0d-b8a8-aed8b2ad1113 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:34.633997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1a79269e-6fbd-4c0d-b8a8-aed8b2ad1113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:34.850057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79269e-6fbd-4c0d-b8a8-aed8b2ad1113 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:34.939954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:35.020509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dddc62-9821-4c33-8755-30e3641375ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:35.021978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dddc62-9821-4c33-8755-30e3641375ba service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:35.022553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dddc62-9821-4c33-8755-30e3641375ba service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:35.023170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dddc62-9821-4c33-8755-30e3641375ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:35.027238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c6dddc62-9821-4c33-8755-30e3641375ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:35.186518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dddc62-9821-4c33-8755-30e3641375ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:35.188220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dddc62-9821-4c33-8755-30e3641375ba service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:35.210219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 720/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:44.331399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=79 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:45.207527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=64 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:48.630258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2ebc2-1b5d-453c-a864-5821f681ba2d None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:48.630844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2ebc2-1b5d-453c-a864-5821f681ba2d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:48.630844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2ebc2-1b5d-453c-a864-5821f681ba2d None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:48.632205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 690/1410] 10.222.0.52 () {54 vars in 918 bytes} [Wed Sep 28 08:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:48.638046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97650419-d7af-4772-b4ec-6abf5f8bf030 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:48.638454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97650419-d7af-4772-b4ec-6abf5f8bf030 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:48.638632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97650419-d7af-4772-b4ec-6abf5f8bf030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:48.715787 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-97650419-d7af-4772-b4ec-6abf5f8bf030 None None] MFA Rules not processed for user `6b40ee263df74e58bf3f958c01eb742f`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:48.717014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-97650419-d7af-4772-b4ec-6abf5f8bf030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:48.826250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 721/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Sep 28 08:31:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:48.840792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e3efa-3e0c-4ec7-a457-25c2c4b0a7a4 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:48.841140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e3efa-3e0c-4ec7-a457-25c2c4b0a7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:48.841423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e3efa-3e0c-4ec7-a457-25c2c4b0a7a4 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:48.842828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 691/1412] 10.222.0.52 () {54 vars in 918 bytes} [Wed Sep 28 08:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:48.847816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045d0f1-fad7-477f-91b9-5a08c619ce4c None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:48.848106 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045d0f1-fad7-477f-91b9-5a08c619ce4c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:48.848387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045d0f1-fad7-477f-91b9-5a08c619ce4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:48.925296 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:48.928205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-3045d0f1-fad7-477f-91b9-5a08c619ce4c None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:48.930074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3045d0f1-fad7-477f-91b9-5a08c619ce4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:49.053920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 722/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Sep 28 08:31:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:51.710104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fe9be-ddf8-43f0-a6dc-fbf5cb4cab52 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:51.710676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fe9be-ddf8-43f0-a6dc-fbf5cb4cab52 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:51.710676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fe9be-ddf8-43f0-a6dc-fbf5cb4cab52 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:51.711978 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 692/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:51.718105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71907a71-26ae-46db-8e3c-17ebce755531 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:51.718572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71907a71-26ae-46db-8e3c-17ebce755531 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:51.718695 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71907a71-26ae-46db-8e3c-17ebce755531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:51.785831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-71907a71-26ae-46db-8e3c-17ebce755531 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:51.787000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-71907a71-26ae-46db-8e3c-17ebce755531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:51.845283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:51.888162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 723/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:52.184769 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fe9be-ddf8-43f0-a6dc-fbf5cb4cab52 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:52.186389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-db8fe9be-ddf8-43f0-a6dc-fbf5cb4cab52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:52.348046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fe9be-ddf8-43f0-a6dc-fbf5cb4cab52 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:52.425409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:52.467026 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca7bab-ec00-4f75-82a8-431dad818533 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:52.467636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca7bab-ec00-4f75-82a8-431dad818533 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:52.467916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca7bab-ec00-4f75-82a8-431dad818533 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:52.468185 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca7bab-ec00-4f75-82a8-431dad818533 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:52.469819 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e1ca7bab-ec00-4f75-82a8-431dad818533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:52.567128 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca7bab-ec00-4f75-82a8-431dad818533 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:52.568770 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca7bab-ec00-4f75-82a8-431dad818533 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:52.595107 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 693/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:31:53.753687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5614-dbd2-41a2-97c5-feef0ec0b3fc None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:53.754287 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5614-dbd2-41a2-97c5-feef0ec0b3fc None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:53.754287 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5614-dbd2-41a2-97c5-feef0ec0b3fc None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:53.755566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 724/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 08:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:53.762300 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a739d-f90a-4ca9-a0c3-fa5169cbb953 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:53.762812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a739d-f90a-4ca9-a0c3-fa5169cbb953 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:53.762930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a739d-f90a-4ca9-a0c3-fa5169cbb953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:53.842350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-998a739d-f90a-4ca9-a0c3-fa5169cbb953 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:53.843487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-998a739d-f90a-4ca9-a0c3-fa5169cbb953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:53.873346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:53.946683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 694/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 08:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:54.113423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d5614-dbd2-41a2-97c5-feef0ec0b3fc None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:54.114878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-521d5614-dbd2-41a2-97c5-feef0ec0b3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:54.272080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d5614-dbd2-41a2-97c5-feef0ec0b3fc None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:54.314637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:54.400341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e436d9b-4eee-47cd-8ea2-2f0234066e5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:54.400855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e436d9b-4eee-47cd-8ea2-2f0234066e5e service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:54.401158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e436d9b-4eee-47cd-8ea2-2f0234066e5e service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:54.401429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e436d9b-4eee-47cd-8ea2-2f0234066e5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:54.403027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7e436d9b-4eee-47cd-8ea2-2f0234066e5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:54.498665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e436d9b-4eee-47cd-8ea2-2f0234066e5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:54.500295 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e436d9b-4eee-47cd-8ea2-2f0234066e5e service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:54.519892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 725/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:31:55.719430 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c6df6-3da4-423b-8f47-9e1a79a95d63 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:55.720235 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c6df6-3da4-423b-8f47-9e1a79a95d63 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:55.720235 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c6df6-3da4-423b-8f47-9e1a79a95d63 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:55.721429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 695/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 08:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:55.729286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8355b1-88c2-4a09-9b56-cb26ed0bfb54 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:55.729835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8355b1-88c2-4a09-9b56-cb26ed0bfb54 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:55.729952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8355b1-88c2-4a09-9b56-cb26ed0bfb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:55.792138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:55.816587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-ae8355b1-88c2-4a09-9b56-cb26ed0bfb54 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:55.817949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ae8355b1-88c2-4a09-9b56-cb26ed0bfb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:55.924484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 726/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 08:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:56.079093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c6df6-3da4-423b-8f47-9e1a79a95d63 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:56.080579 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-910c6df6-3da4-423b-8f47-9e1a79a95d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:56.244292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c6df6-3da4-423b-8f47-9e1a79a95d63 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:56.257751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:56.371313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095787cf-66e2-4c71-99f3-b7ac2b23cbe1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:56.371838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095787cf-66e2-4c71-99f3-b7ac2b23cbe1 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:56.372113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095787cf-66e2-4c71-99f3-b7ac2b23cbe1 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:56.372379 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095787cf-66e2-4c71-99f3-b7ac2b23cbe1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:56.374048 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-095787cf-66e2-4c71-99f3-b7ac2b23cbe1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:56.475567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095787cf-66e2-4c71-99f3-b7ac2b23cbe1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:56.477208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095787cf-66e2-4c71-99f3-b7ac2b23cbe1 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:56.497030 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 696/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:57.666597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8f4db-1ae7-4b0c-b963-245be2898b3a None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:57.667273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8f4db-1ae7-4b0c-b963-245be2898b3a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:57.667273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8f4db-1ae7-4b0c-b963-245be2898b3a None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:57.668700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 727/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 08:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:57.675969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3435237-2eda-488d-ad35-f7476c6c342d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:57.676531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3435237-2eda-488d-ad35-f7476c6c342d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:57.676531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3435237-2eda-488d-ad35-f7476c6c342d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:57.698368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:57.759914 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-e3435237-2eda-488d-ad35-f7476c6c342d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:57.761151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e3435237-2eda-488d-ad35-f7476c6c342d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:57.868883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 697/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 08:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:58.027075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8f4db-1ae7-4b0c-b963-245be2898b3a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:31:58.028641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-76e8f4db-1ae7-4b0c-b963-245be2898b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:58.152111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:58.189773 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8f4db-1ae7-4b0c-b963-245be2898b3a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:31:58.310140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc4570-5cd2-45d2-baa2-79833e9a6818 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:31:58.310762 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc4570-5cd2-45d2-baa2-79833e9a6818 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:58.311043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc4570-5cd2-45d2-baa2-79833e9a6818 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:58.311315 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc4570-5cd2-45d2-baa2-79833e9a6818 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:58.312985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-eefc4570-5cd2-45d2-baa2-79833e9a6818 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:58.409065 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:31:58.423871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc4570-5cd2-45d2-baa2-79833e9a6818 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:31:58.425502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc4570-5cd2-45d2-baa2-79833e9a6818 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:31:58.445604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 728/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:31:59.620941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d46205-b912-4af4-bb1d-5a791212a228 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:59.621572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d46205-b912-4af4-bb1d-5a791212a228 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:59.621572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d46205-b912-4af4-bb1d-5a791212a228 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:59.622864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 698/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 08:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:31:59.629650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688260a8-e2f4-4924-86cd-64ba28dc5832 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:31:59.630182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688260a8-e2f4-4924-86cd-64ba28dc5832 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:31:59.630182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688260a8-e2f4-4924-86cd-64ba28dc5832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:31:59.730821 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-688260a8-e2f4-4924-86cd-64ba28dc5832 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:31:59.732083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-688260a8-e2f4-4924-86cd-64ba28dc5832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:31:59.846343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 729/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 08:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:31:59.999666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d46205-b912-4af4-bb1d-5a791212a228 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:00.002071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-18d46205-b912-4af4-bb1d-5a791212a228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:00.098947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:00.167589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d46205-b912-4af4-bb1d-5a791212a228 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:00.288503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb464d-5886-4d32-ae79-75b3ce9d77df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:00.289024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb464d-5886-4d32-ae79-75b3ce9d77df service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:00.289334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb464d-5886-4d32-ae79-75b3ce9d77df service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:00.289607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb464d-5886-4d32-ae79-75b3ce9d77df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:00.291862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f3bb464d-5886-4d32-ae79-75b3ce9d77df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:00.344224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:00.389203 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb464d-5886-4d32-ae79-75b3ce9d77df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:00.390832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb464d-5886-4d32-ae79-75b3ce9d77df service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:00.410351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 699/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:31:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:01.498895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53674ce-4e3b-4c77-bb6e-e74d1a7a7745 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:01.499478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53674ce-4e3b-4c77-bb6e-e74d1a7a7745 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:01.499478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53674ce-4e3b-4c77-bb6e-e74d1a7a7745 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:01.500781 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 730/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 08:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:01.507650 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b951e-efa4-48c8-9fa8-53961c4aeaf7 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:01.508417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b951e-efa4-48c8-9fa8-53961c4aeaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:01.508417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b951e-efa4-48c8-9fa8-53961c4aeaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:01.598703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-1c8b951e-efa4-48c8-9fa8-53961c4aeaf7 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:01.599934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1c8b951e-efa4-48c8-9fa8-53961c4aeaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:01.702373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 700/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 08:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:01.858070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53674ce-4e3b-4c77-bb6e-e74d1a7a7745 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:01.859676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d53674ce-4e3b-4c77-bb6e-e74d1a7a7745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:01.917610 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:02.022041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53674ce-4e3b-4c77-bb6e-e74d1a7a7745 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:02.149188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c0906-1ac8-461c-a734-72dce2d98915 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:02.149729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c0906-1ac8-461c-a734-72dce2d98915 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:02.150009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c0906-1ac8-461c-a734-72dce2d98915 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:02.150281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c0906-1ac8-461c-a734-72dce2d98915 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:02.151934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-141c0906-1ac8-461c-a734-72dce2d98915 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:02.159894 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:02.374306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c0906-1ac8-461c-a734-72dce2d98915 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:02.376037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c0906-1ac8-461c-a734-72dce2d98915 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:02.394557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 731/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:03.486842 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632977e5-63fc-4eb2-a507-2c11c02cead1 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:03.487366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632977e5-63fc-4eb2-a507-2c11c02cead1 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:03.487366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632977e5-63fc-4eb2-a507-2c11c02cead1 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:03.488734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 701/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 08:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:03.496029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d4fe5-5839-48fe-b3b9-56529d385c3e None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:03.496507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d4fe5-5839-48fe-b3b9-56529d385c3e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:03.496606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d4fe5-5839-48fe-b3b9-56529d385c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:03.576672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-114d4fe5-5839-48fe-b3b9-56529d385c3e None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:03.577984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-114d4fe5-5839-48fe-b3b9-56529d385c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:03.652405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:03.683918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 732/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 08:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:03.833902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632977e5-63fc-4eb2-a507-2c11c02cead1 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:03.835339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-632977e5-63fc-4eb2-a507-2c11c02cead1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:03.853760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:03.982698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632977e5-63fc-4eb2-a507-2c11c02cead1 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:04.082226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:04.120154 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5333b-fa73-44b7-9304-ae7c4c5cfa50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:04.120677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5333b-fa73-44b7-9304-ae7c4c5cfa50 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:04.120959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5333b-fa73-44b7-9304-ae7c4c5cfa50 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:04.121268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5333b-fa73-44b7-9304-ae7c4c5cfa50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:04.122877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-fca5333b-fa73-44b7-9304-ae7c4c5cfa50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:04.215489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5333b-fa73-44b7-9304-ae7c4c5cfa50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:04.217121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5333b-fa73-44b7-9304-ae7c4c5cfa50 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:04.234792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 702/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:05.305216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d933b70-e6fe-4156-9fee-c0fdacba3238 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:05.306480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d933b70-e6fe-4156-9fee-c0fdacba3238 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:05.306480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d933b70-e6fe-4156-9fee-c0fdacba3238 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:05.307715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 733/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 08:32:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:05.314760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c286007-126e-4d30-b600-46b1a1dac633 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:05.315290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c286007-126e-4d30-b600-46b1a1dac633 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:05.315290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c286007-126e-4d30-b600-46b1a1dac633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:05.395375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-5c286007-126e-4d30-b600-46b1a1dac633 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:05.396510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5c286007-126e-4d30-b600-46b1a1dac633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:05.439377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:05.496257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 703/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 08:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:05.650131 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d933b70-e6fe-4156-9fee-c0fdacba3238 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:05.651714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5d933b70-e6fe-4156-9fee-c0fdacba3238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:05.804983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d933b70-e6fe-4156-9fee-c0fdacba3238 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:05.860436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:05.918393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cfda4-0fc6-48cb-946a-18962c2309bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:05.918917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cfda4-0fc6-48cb-946a-18962c2309bf service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:05.919186 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cfda4-0fc6-48cb-946a-18962c2309bf service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:05.919449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cfda4-0fc6-48cb-946a-18962c2309bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:05.921007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3c4cfda4-0fc6-48cb-946a-18962c2309bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:06.008679 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4cfda4-0fc6-48cb-946a-18962c2309bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:06.010279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4cfda4-0fc6-48cb-946a-18962c2309bf service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:06.028532 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 734/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:07.113016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dea2c-694c-4e42-bb1e-bb5a296a2e66 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:07.115438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dea2c-694c-4e42-bb1e-bb5a296a2e66 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:07.115438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dea2c-694c-4e42-bb1e-bb5a296a2e66 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:07.115438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 704/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 08:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:07.123747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60cbbc-84aa-49d6-9a4a-0f3e20d2efc2 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:07.124332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60cbbc-84aa-49d6-9a4a-0f3e20d2efc2 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:07.124332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60cbbc-84aa-49d6-9a4a-0f3e20d2efc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:07.211475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2e60cbbc-84aa-49d6-9a4a-0f3e20d2efc2 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:07.212694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2e60cbbc-84aa-49d6-9a4a-0f3e20d2efc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:07.214810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:07.313917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 735/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 08:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:07.465277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dea2c-694c-4e42-bb1e-bb5a296a2e66 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:07.466795 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-357dea2c-694c-4e42-bb1e-bb5a296a2e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:07.618424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dea2c-694c-4e42-bb1e-bb5a296a2e66 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:07.644868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:07.732593 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30f4e5-eb4f-49b3-aa19-c926b52c567d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:07.733102 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30f4e5-eb4f-49b3-aa19-c926b52c567d service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:07.733399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30f4e5-eb4f-49b3-aa19-c926b52c567d service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:07.733673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30f4e5-eb4f-49b3-aa19-c926b52c567d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:07.735283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9f30f4e5-eb4f-49b3-aa19-c926b52c567d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:07.824805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30f4e5-eb4f-49b3-aa19-c926b52c567d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:07.826457 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30f4e5-eb4f-49b3-aa19-c926b52c567d service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:07.844527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 705/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:09.963478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1551803-e0a8-497f-a352-ad1652b0541c None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:09.964061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1551803-e0a8-497f-a352-ad1652b0541c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:09.964061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1551803-e0a8-497f-a352-ad1652b0541c None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:09.965407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 736/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:09.970841 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac3acc-fb8d-4d85-9258-a7f99f5627ed None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:09.971286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac3acc-fb8d-4d85-9258-a7f99f5627ed None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:09.971394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac3acc-fb8d-4d85-9258-a7f99f5627ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:10.012982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:10.038582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-23ac3acc-fb8d-4d85-9258-a7f99f5627ed None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:10.039802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-23ac3acc-fb8d-4d85-9258-a7f99f5627ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:10.136570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 706/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:10.431029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1551803-e0a8-497f-a352-ad1652b0541c None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:10.433199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a1551803-e0a8-497f-a352-ad1652b0541c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:10.553703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1551803-e0a8-497f-a352-ad1652b0541c None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:10.588458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:10.632630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816cdd2-b414-489d-b5ab-ca79df767454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:10.633267 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816cdd2-b414-489d-b5ab-ca79df767454 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:10.633544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816cdd2-b414-489d-b5ab-ca79df767454 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:10.633810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816cdd2-b414-489d-b5ab-ca79df767454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:10.635390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1816cdd2-b414-489d-b5ab-ca79df767454 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:10.744364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816cdd2-b414-489d-b5ab-ca79df767454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:10.745823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816cdd2-b414-489d-b5ab-ca79df767454 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:10.763958 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 737/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:13.069827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf651f7-3e2f-455a-aa0f-9dabf97771c2 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:13.070417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf651f7-3e2f-455a-aa0f-9dabf97771c2 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:13.070417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf651f7-3e2f-455a-aa0f-9dabf97771c2 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:13.071858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 707/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:13.077192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6708e9a-0661-4a57-9523-4a11621e3227 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:13.077596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6708e9a-0661-4a57-9523-4a11621e3227 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:13.077747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6708e9a-0661-4a57-9523-4a11621e3227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:13.154889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b6708e9a-0661-4a57-9523-4a11621e3227 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:13.156114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b6708e9a-0661-4a57-9523-4a11621e3227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:13.169692 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:13.252494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 738/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:13.549606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf651f7-3e2f-455a-aa0f-9dabf97771c2 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:13.551143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ccf651f7-3e2f-455a-aa0f-9dabf97771c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:13.657485 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf651f7-3e2f-455a-aa0f-9dabf97771c2 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:13.670684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:13.734097 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ffb1b9-0be8-4628-bf63-a3cefdd5f6bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:13.734696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffb1b9-0be8-4628-bf63-a3cefdd5f6bb service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:13.734969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffb1b9-0be8-4628-bf63-a3cefdd5f6bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:13.735233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffb1b9-0be8-4628-bf63-a3cefdd5f6bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:13.736802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f5ffb1b9-0be8-4628-bf63-a3cefdd5f6bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:13.853043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffb1b9-0be8-4628-bf63-a3cefdd5f6bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:13.854444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ffb1b9-0be8-4628-bf63-a3cefdd5f6bb service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:13.873151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 708/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:16.362006 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfd4f3-2cf2-4646-aae8-24ab30cd4dfa None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:16.362594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfd4f3-2cf2-4646-aae8-24ab30cd4dfa None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:16.362594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfd4f3-2cf2-4646-aae8-24ab30cd4dfa None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:16.363932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 739/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:16.370083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f516994-b855-44e1-87e4-6a1878c4dc1f None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:16.370532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f516994-b855-44e1-87e4-6a1878c4dc1f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:16.370638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f516994-b855-44e1-87e4-6a1878c4dc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:16.417753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:16.436178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-9f516994-b855-44e1-87e4-6a1878c4dc1f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:16.437461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9f516994-b855-44e1-87e4-6a1878c4dc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:16.534310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 709/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:16.822365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfd4f3-2cf2-4646-aae8-24ab30cd4dfa None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:16.823902 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4dcfd4f3-2cf2-4646-aae8-24ab30cd4dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:16.932246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfd4f3-2cf2-4646-aae8-24ab30cd4dfa None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:16.973062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:17.005608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44ddbb-9223-481b-9777-a15d760cef1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:17.006209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44ddbb-9223-481b-9777-a15d760cef1c service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:17.006481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44ddbb-9223-481b-9777-a15d760cef1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:17.006743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44ddbb-9223-481b-9777-a15d760cef1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:17.008317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-fa44ddbb-9223-481b-9777-a15d760cef1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:17.116087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44ddbb-9223-481b-9777-a15d760cef1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:17.117447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44ddbb-9223-481b-9777-a15d760cef1c service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:17.135567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 740/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:19.440727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf56b4-4a0d-4133-8029-b7211b1ccade None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:19.441271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf56b4-4a0d-4133-8029-b7211b1ccade None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:19.441271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf56b4-4a0d-4133-8029-b7211b1ccade None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:19.442557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 710/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:19.447855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59b439-63d6-4f71-afec-4a5c46830587 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:19.448279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59b439-63d6-4f71-afec-4a5c46830587 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:19.448411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59b439-63d6-4f71-afec-4a5c46830587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:19.516777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-6c59b439-63d6-4f71-afec-4a5c46830587 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:19.518005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-6c59b439-63d6-4f71-afec-4a5c46830587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:19.544636 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:19.624685 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 741/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:19.914338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf56b4-4a0d-4133-8029-b7211b1ccade None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:19.916420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3caf56b4-4a0d-4133-8029-b7211b1ccade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:20.021800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf56b4-4a0d-4133-8029-b7211b1ccade None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:20.042472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:20.095883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad3521-10c2-4c55-bb32-9dca86eba2a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:20.096482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad3521-10c2-4c55-bb32-9dca86eba2a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:20.096760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad3521-10c2-4c55-bb32-9dca86eba2a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:20.097030 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad3521-10c2-4c55-bb32-9dca86eba2a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:20.098652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c0ad3521-10c2-4c55-bb32-9dca86eba2a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:20.208331 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad3521-10c2-4c55-bb32-9dca86eba2a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:20.209799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad3521-10c2-4c55-bb32-9dca86eba2a2 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:20.228082 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 711/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:32:22.578151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e1927-d200-44f2-ad71-2ce19f497a93 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:22.578683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e1927-d200-44f2-ad71-2ce19f497a93 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:22.578683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e1927-d200-44f2-ad71-2ce19f497a93 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:22.580044 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 742/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:22.585597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a9b96-6e84-4785-be41-e2a8039f4090 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:22.586055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a9b96-6e84-4785-be41-e2a8039f4090 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:22.586162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a9b96-6e84-4785-be41-e2a8039f4090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:22.652210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:22.655029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-346a9b96-6e84-4785-be41-e2a8039f4090 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:22.656907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-346a9b96-6e84-4785-be41-e2a8039f4090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:22.754562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 712/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:23.056394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e1927-d200-44f2-ad71-2ce19f497a93 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:23.057969 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2f9e1927-d200-44f2-ad71-2ce19f497a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:23.163653 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e1927-d200-44f2-ad71-2ce19f497a93 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:23.214126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:23.239133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd91fc-8d7a-457b-9ba9-046a655c5075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:23.239735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd91fc-8d7a-457b-9ba9-046a655c5075 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:23.240014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd91fc-8d7a-457b-9ba9-046a655c5075 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:23.240288 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd91fc-8d7a-457b-9ba9-046a655c5075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:23.242030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-42cd91fc-8d7a-457b-9ba9-046a655c5075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:23.351074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd91fc-8d7a-457b-9ba9-046a655c5075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:23.352424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd91fc-8d7a-457b-9ba9-046a655c5075 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:23.370656 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 743/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:25.572590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad10f-ebd9-4ef1-8329-e1355fd20dad None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:25.573153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad10f-ebd9-4ef1-8329-e1355fd20dad None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:25.573153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad10f-ebd9-4ef1-8329-e1355fd20dad None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:25.574493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 713/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:25.579905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743e66a-cb33-4b3f-9040-7d65d2d75228 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:25.580296 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743e66a-cb33-4b3f-9040-7d65d2d75228 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:25.580457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743e66a-cb33-4b3f-9040-7d65d2d75228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:25.648026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-7743e66a-cb33-4b3f-9040-7d65d2d75228 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:25.649830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7743e66a-cb33-4b3f-9040-7d65d2d75228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:25.688555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:25.759063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 744/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:26.056023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ad10f-ebd9-4ef1-8329-e1355fd20dad None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:26.058385 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ed4ad10f-ebd9-4ef1-8329-e1355fd20dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:26.161823 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ad10f-ebd9-4ef1-8329-e1355fd20dad None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:26.190915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:26.236548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e33db-f4ae-431a-8465-c7b25f7ec764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:26.237183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e33db-f4ae-431a-8465-c7b25f7ec764 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:26.237469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e33db-f4ae-431a-8465-c7b25f7ec764 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:26.237741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e33db-f4ae-431a-8465-c7b25f7ec764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:26.239325 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-334e33db-f4ae-431a-8465-c7b25f7ec764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:26.348510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e33db-f4ae-431a-8465-c7b25f7ec764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:26.349899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e33db-f4ae-431a-8465-c7b25f7ec764 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:26.368676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 714/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:28.591805 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2da75e-7ed1-4b4f-8d86-99791be3b146 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:28.592372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2da75e-7ed1-4b4f-8d86-99791be3b146 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:28.592372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2da75e-7ed1-4b4f-8d86-99791be3b146 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:28.593747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 745/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:28.599287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80a9d3-c52e-4a29-b61b-169ea8d07c42 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:28.599718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80a9d3-c52e-4a29-b61b-169ea8d07c42 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:28.599843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80a9d3-c52e-4a29-b61b-169ea8d07c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:28.668536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ca80a9d3-c52e-4a29-b61b-169ea8d07c42 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:28.669818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ca80a9d3-c52e-4a29-b61b-169ea8d07c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:28.678446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:28.766762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 715/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:32:29.055972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2da75e-7ed1-4b4f-8d86-99791be3b146 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:29.057565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-bb2da75e-7ed1-4b4f-8d86-99791be3b146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:29.160939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2da75e-7ed1-4b4f-8d86-99791be3b146 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:29.217457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:29.235783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e35402-9826-4def-8ad2-5e72ee2fc710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:29.236391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e35402-9826-4def-8ad2-5e72ee2fc710 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:29.236673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e35402-9826-4def-8ad2-5e72ee2fc710 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:29.236946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e35402-9826-4def-8ad2-5e72ee2fc710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:29.238695 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-13e35402-9826-4def-8ad2-5e72ee2fc710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:29.359045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e35402-9826-4def-8ad2-5e72ee2fc710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:29.360393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e35402-9826-4def-8ad2-5e72ee2fc710 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:29.379478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 746/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:31.622595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653eed9-0d71-4c9a-b508-f4c2448e16ff None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:31.623263 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653eed9-0d71-4c9a-b508-f4c2448e16ff None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:31.623263 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653eed9-0d71-4c9a-b508-f4c2448e16ff None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:31.624483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 716/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:31.629775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44f768-aba5-45fd-acb0-e2b98f390b45 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:31.630232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44f768-aba5-45fd-acb0-e2b98f390b45 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:31.630337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44f768-aba5-45fd-acb0-e2b98f390b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:31.697072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5e44f768-aba5-45fd-acb0-e2b98f390b45 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:31.698330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5e44f768-aba5-45fd-acb0-e2b98f390b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:31.737805 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:31.797250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 747/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:32.087037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653eed9-0d71-4c9a-b508-f4c2448e16ff None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:32.088584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c653eed9-0d71-4c9a-b508-f4c2448e16ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:32.196295 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653eed9-0d71-4c9a-b508-f4c2448e16ff None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:32.238287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:32.275615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb823d3-6291-456f-8e48-29f0e082daca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:32.276221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb823d3-6291-456f-8e48-29f0e082daca service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:32.276502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb823d3-6291-456f-8e48-29f0e082daca service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:32.276776 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb823d3-6291-456f-8e48-29f0e082daca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:32.278405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8bb823d3-6291-456f-8e48-29f0e082daca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:32.390520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb823d3-6291-456f-8e48-29f0e082daca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:32.391877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb823d3-6291-456f-8e48-29f0e082daca service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:32.410848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 717/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:34.622120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcccf55-32e3-408f-b52f-bfcba788e783 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:34.622667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcccf55-32e3-408f-b52f-bfcba788e783 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:34.622667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcccf55-32e3-408f-b52f-bfcba788e783 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:34.624023 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 748/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:34.629654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a8434-4d55-47ee-a54a-6eb3c61ca551 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:34.630077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a8434-4d55-47ee-a54a-6eb3c61ca551 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:34.630200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a8434-4d55-47ee-a54a-6eb3c61ca551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:34.701275 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-240a8434-4d55-47ee-a54a-6eb3c61ca551 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:34.702499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-240a8434-4d55-47ee-a54a-6eb3c61ca551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:34.720878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:34.798655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 718/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:35.094136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcccf55-32e3-408f-b52f-bfcba788e783 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:35.095666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8bcccf55-32e3-408f-b52f-bfcba788e783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:35.200503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcccf55-32e3-408f-b52f-bfcba788e783 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:35.267218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:35.274536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cedb82-36ce-40c5-8dd8-e9d0802747ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:35.275142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cedb82-36ce-40c5-8dd8-e9d0802747ff service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:35.275421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cedb82-36ce-40c5-8dd8-e9d0802747ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:35.275693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cedb82-36ce-40c5-8dd8-e9d0802747ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:35.277302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-83cedb82-36ce-40c5-8dd8-e9d0802747ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:35.391704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cedb82-36ce-40c5-8dd8-e9d0802747ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:35.393039 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cedb82-36ce-40c5-8dd8-e9d0802747ff service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:35.410911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 749/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:37.574608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a48f33-2861-49d7-9304-bb25a9b528bd None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:37.575218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a48f33-2861-49d7-9304-bb25a9b528bd None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:37.575218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a48f33-2861-49d7-9304-bb25a9b528bd None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:37.576483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 719/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:37.582356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9ecca-ad8e-413e-bd2c-5f114bc1c809 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:37.582788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9ecca-ad8e-413e-bd2c-5f114bc1c809 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:37.582995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9ecca-ad8e-413e-bd2c-5f114bc1c809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:37.651583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-76f9ecca-ad8e-413e-bd2c-5f114bc1c809 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:37.652791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-76f9ecca-ad8e-413e-bd2c-5f114bc1c809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:37.706349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:37.766902 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 750/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:32:38.067096 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a48f33-2861-49d7-9304-bb25a9b528bd None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:38.068634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-06a48f33-2861-49d7-9304-bb25a9b528bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:38.173995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a48f33-2861-49d7-9304-bb25a9b528bd None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:38.224840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:38.251029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdbacc-9b8f-4163-a897-c9a6d410a69e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:38.251630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdbacc-9b8f-4163-a897-c9a6d410a69e service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:38.251909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdbacc-9b8f-4163-a897-c9a6d410a69e service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:38.252209 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdbacc-9b8f-4163-a897-c9a6d410a69e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:38.253848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4bcdbacc-9b8f-4163-a897-c9a6d410a69e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:38.362986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdbacc-9b8f-4163-a897-c9a6d410a69e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:38.364356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdbacc-9b8f-4163-a897-c9a6d410a69e service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:38.386279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 720/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:32:47.765412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=22 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:48.382119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=66 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:58.612047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcf5d9-c993-4e12-ac22-73722b1ae1e8 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:58.612601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcf5d9-c993-4e12-ac22-73722b1ae1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:58.612601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcf5d9-c993-4e12-ac22-73722b1ae1e8 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:58.613964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 751/1471] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:58.619836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f34b4a-3003-46bb-a465-0714272073e4 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:58.620254 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f34b4a-3003-46bb-a465-0714272073e4 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:58.620373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f34b4a-3003-46bb-a465-0714272073e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:58.686353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-08f34b4a-3003-46bb-a465-0714272073e4 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:58.687561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-08f34b4a-3003-46bb-a465-0714272073e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:58.785780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 721/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:32:59.090336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b83049-494c-4fdc-abf6-3a0e98a7a05c None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:59.090628 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b83049-494c-4fdc-abf6-3a0e98a7a05c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:59.090935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b83049-494c-4fdc-abf6-3a0e98a7a05c None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:59.092268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 752/1473] 10.222.0.52 () {54 vars in 962 bytes} [Wed Sep 28 08:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:32:59.097257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dee67d-8539-4548-a956-79c24b4814b2 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:59.097548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dee67d-8539-4548-a956-79c24b4814b2 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:59.097820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dee67d-8539-4548-a956-79c24b4814b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:59.181943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-e2dee67d-8539-4548-a956-79c24b4814b2 None None] MFA Rules not processed for user `ff22c2632c614fca8008feab93c8fa2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:32:59.183157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e2dee67d-8539-4548-a956-79c24b4814b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:59.203755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:59.259155 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 722/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Sep 28 08:32:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:32:59.266919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b83049-494c-4fdc-abf6-3a0e98a7a05c None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:32:59.268419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-68b83049-494c-4fdc-abf6-3a0e98a7a05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:59.372871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b83049-494c-4fdc-abf6-3a0e98a7a05c None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:32:59.455620 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac3565-79ef-48e2-9e8e-db5c231266c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff22c2632c614fca8008feab93c8fa2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:32:59.456230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac3565-79ef-48e2-9e8e-db5c231266c8 service glance] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:32:59.456508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac3565-79ef-48e2-9e8e-db5c231266c8 service glance] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:32:59.456816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac3565-79ef-48e2-9e8e-db5c231266c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:32:59.458738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f9ac3565-79ef-48e2-9e8e-db5c231266c8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:32:59.513401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:32:59.570098 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac3565-79ef-48e2-9e8e-db5c231266c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:32:59.572011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac3565-79ef-48e2-9e8e-db5c231266c8 service glance] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:32:59.595386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 753/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:33:01.844827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65624b-f010-443f-af99-863e0f3b3c25 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:01.845416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65624b-f010-443f-af99-863e0f3b3c25 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:01.845416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65624b-f010-443f-af99-863e0f3b3c25 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:01.846742 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 723/1476] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:33:01.852197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402269d-577a-4d12-a06c-2f4cc03fb36d None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:01.852681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402269d-577a-4d12-a06c-2f4cc03fb36d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:01.852786 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402269d-577a-4d12-a06c-2f4cc03fb36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:01.918667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-a402269d-577a-4d12-a06c-2f4cc03fb36d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:33:01.919868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a402269d-577a-4d12-a06c-2f4cc03fb36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:02.014637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 754/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:33:02.298161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65624b-f010-443f-af99-863e0f3b3c25 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:33:02.299684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-af65624b-f010-443f-af99-863e0f3b3c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:02.426758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65624b-f010-443f-af99-863e0f3b3c25 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:33:02.478406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:33:02.526645 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c73fe4-1d53-41bc-9853-794139b12566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:33:02.527258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c73fe4-1d53-41bc-9853-794139b12566 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:02.527533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c73fe4-1d53-41bc-9853-794139b12566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:02.527801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c73fe4-1d53-41bc-9853-794139b12566 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:02.528441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c73fe4-1d53-41bc-9853-794139b12566 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:33:02.530256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: WARNING py.warnings [None req-18c73fe4-1d53-41bc-9853-794139b12566 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:33:02.530256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: warnings.warn(msg) Sep 28 08:33:02.530256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: Sep 28 08:33:02.531413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c73fe4-1d53-41bc-9853-794139b12566 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:33:02.551571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 724/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 08:33:02 2022] GET /identity/v3/projects => generated 2553 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:33:04.764409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35d0fa-217b-44ac-9c38-3878644845a7 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:04.764968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35d0fa-217b-44ac-9c38-3878644845a7 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:04.764968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35d0fa-217b-44ac-9c38-3878644845a7 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:04.766336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 755/1479] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:33:04.771643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459f182-d438-46de-8da3-f1fe389aa1ca None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:04.772103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459f182-d438-46de-8da3-f1fe389aa1ca None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:04.772208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459f182-d438-46de-8da3-f1fe389aa1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:04.963024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-6459f182-d438-46de-8da3-f1fe389aa1ca None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:33:04.964260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6459f182-d438-46de-8da3-f1fe389aa1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:05.060391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 725/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:33:05.359566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35d0fa-217b-44ac-9c38-3878644845a7 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:33:05.361168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ab35d0fa-217b-44ac-9c38-3878644845a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:05.371543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:33:05.518105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35d0fa-217b-44ac-9c38-3878644845a7 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:33:05.599595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:33:05.634142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9b8ea-b362-492c-b1f8-7c7c7430b223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:33:05.634753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9b8ea-b362-492c-b1f8-7c7c7430b223 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:05.635033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9b8ea-b362-492c-b1f8-7c7c7430b223 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:05.635307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9b8ea-b362-492c-b1f8-7c7c7430b223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:05.636889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5ae9b8ea-b362-492c-b1f8-7c7c7430b223 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:05.727513 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9b8ea-b362-492c-b1f8-7c7c7430b223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:33:05.729141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9b8ea-b362-492c-b1f8-7c7c7430b223 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:33:05.747593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 756/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:33:08.075647 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f5b7-2cb2-402e-a269-a2d5ce4a6a7c None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:08.076183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f5b7-2cb2-402e-a269-a2d5ce4a6a7c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:08.076183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f5b7-2cb2-402e-a269-a2d5ce4a6a7c None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:08.077552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 726/1482] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:33:08.082783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d22f2f-445a-4cb6-a4d5-a38301693494 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:08.083222 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d22f2f-445a-4cb6-a4d5-a38301693494 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:08.083325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d22f2f-445a-4cb6-a4d5-a38301693494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:08.150074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-c7d22f2f-445a-4cb6-a4d5-a38301693494 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:33:08.151326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c7d22f2f-445a-4cb6-a4d5-a38301693494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:08.200272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:33:08.253826 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 757/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:33:08.548312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f5b7-2cb2-402e-a269-a2d5ce4a6a7c None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:33:08.549901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1588f5b7-2cb2-402e-a269-a2d5ce4a6a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:08.575751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:33:08.665295 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f5b7-2cb2-402e-a269-a2d5ce4a6a7c None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:33:08.737496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c1481-38ee-4dac-8b46-c1eea7771445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:33:08.738088 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c1481-38ee-4dac-8b46-c1eea7771445 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:08.738362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c1481-38ee-4dac-8b46-c1eea7771445 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:08.738631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c1481-38ee-4dac-8b46-c1eea7771445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:08.740257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-116c1481-38ee-4dac-8b46-c1eea7771445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:08.814170 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:33:08.848747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c1481-38ee-4dac-8b46-c1eea7771445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:33:08.850217 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c1481-38ee-4dac-8b46-c1eea7771445 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:33:08.871614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 727/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:33:11.117617 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48a020-20e5-4099-b8b9-d91b6cef9a29 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:11.118179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48a020-20e5-4099-b8b9-d91b6cef9a29 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:11.118179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48a020-20e5-4099-b8b9-d91b6cef9a29 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:11.119507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 758/1485] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:33:11.124768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1991ebb-ada8-4e59-9d3d-10280d1c528a None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:11.125189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1991ebb-ada8-4e59-9d3d-10280d1c528a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:11.125348 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1991ebb-ada8-4e59-9d3d-10280d1c528a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:11.193319 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-b1991ebb-ada8-4e59-9d3d-10280d1c528a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:33:11.194517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b1991ebb-ada8-4e59-9d3d-10280d1c528a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:11.300070 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 728/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:33:11.613714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48a020-20e5-4099-b8b9-d91b6cef9a29 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:33:11.615260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1d48a020-20e5-4099-b8b9-d91b6cef9a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:11.722292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48a020-20e5-4099-b8b9-d91b6cef9a29 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:33:11.795902 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:33:11.797526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aaec89-baa6-4051-9c6a-507b0f9b31dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:33:11.798226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaec89-baa6-4051-9c6a-507b0f9b31dd service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:11.798509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaec89-baa6-4051-9c6a-507b0f9b31dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:11.798974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaec89-baa6-4051-9c6a-507b0f9b31dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:11.800924 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c4aaec89-baa6-4051-9c6a-507b0f9b31dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:11.914449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aaec89-baa6-4051-9c6a-507b0f9b31dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:33:11.915988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aaec89-baa6-4051-9c6a-507b0f9b31dd service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:33:11.936791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 759/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:33:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:33:14.464308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e9d00-75cd-4e85-9082-2bce45d4a051 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:14.464881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e9d00-75cd-4e85-9082-2bce45d4a051 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:14.464881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e9d00-75cd-4e85-9082-2bce45d4a051 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:14.466231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 729/1488] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:33:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:33:14.471669 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52225f6e-035a-45b0-9876-ab329ec8c16b None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:14.472133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52225f6e-035a-45b0-9876-ab329ec8c16b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:14.472240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52225f6e-035a-45b0-9876-ab329ec8c16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:14.541822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-52225f6e-035a-45b0-9876-ab329ec8c16b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:33:14.543368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-52225f6e-035a-45b0-9876-ab329ec8c16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:14.600286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:33:14.646695 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 760/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:33:14.941645 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e9d00-75cd-4e85-9082-2bce45d4a051 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:33:14.943184 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-496e9d00-75cd-4e85-9082-2bce45d4a051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:14.984022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:33:15.059510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e9d00-75cd-4e85-9082-2bce45d4a051 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:33:15.140318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66823a0-0335-4745-93fd-d66a036a532f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:33:15.140954 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66823a0-0335-4745-93fd-d66a036a532f service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:15.141260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66823a0-0335-4745-93fd-d66a036a532f service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:15.141529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66823a0-0335-4745-93fd-d66a036a532f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:15.143108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a66823a0-0335-4745-93fd-d66a036a532f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:15.231086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:33:15.260390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66823a0-0335-4745-93fd-d66a036a532f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:33:15.261867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66823a0-0335-4745-93fd-d66a036a532f service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:33:15.280786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 730/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:33:17.684606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a0d2f-cb12-493b-a330-6177b6aa0987 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:17.685193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a0d2f-cb12-493b-a330-6177b6aa0987 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:17.685193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a0d2f-cb12-493b-a330-6177b6aa0987 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:17.686545 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 761/1491] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 08:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:33:17.692138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ec02b-30ff-4217-b5cc-fe089494198b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:17.692563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ec02b-30ff-4217-b5cc-fe089494198b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:17.692672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ec02b-30ff-4217-b5cc-fe089494198b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:17.759441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-053ec02b-30ff-4217-b5cc-fe089494198b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:33:17.760643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-053ec02b-30ff-4217-b5cc-fe089494198b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:17.857006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 731/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 08:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:33:18.151551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a0d2f-cb12-493b-a330-6177b6aa0987 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:33:18.153398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-537a0d2f-cb12-493b-a330-6177b6aa0987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:18.261072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a0d2f-cb12-493b-a330-6177b6aa0987 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:33:18.337311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712159f-a703-44c0-ab5a-28d96511a277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:33:18.337917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712159f-a703-44c0-ab5a-28d96511a277 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:33:18.338193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712159f-a703-44c0-ab5a-28d96511a277 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:33:18.338462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712159f-a703-44c0-ab5a-28d96511a277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:33:18.340035 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8712159f-a703-44c0-ab5a-28d96511a277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:33:18.348660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:33:18.451698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712159f-a703-44c0-ab5a-28d96511a277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:33:18.453084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712159f-a703-44c0-ab5a-28d96511a277 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:33:18.471090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 762/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:33:27.855399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=82 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:33:28.468853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=50 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:34:32.668326 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e6472-b604-41d6-8bf7-c574e102e21b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:32.668912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e6472-b604-41d6-8bf7-c574e102e21b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:32.668912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e6472-b604-41d6-8bf7-c574e102e21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:32.745426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-f02e6472-b604-41d6-8bf7-c574e102e21b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:34:32.746653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f02e6472-b604-41d6-8bf7-c574e102e21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:32.847966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 732/1494] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:34:32.856610 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4eb0c3-1cfa-4669-9b8c-e10c0db81f02 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:32.857120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4eb0c3-1cfa-4669-9b8c-e10c0db81f02 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:32.857231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4eb0c3-1cfa-4669-9b8c-e10c0db81f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:32.934968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-0e4eb0c3-1cfa-4669-9b8c-e10c0db81f02 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:34:32.936209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-0e4eb0c3-1cfa-4669-9b8c-e10c0db81f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:33.033188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 763/1495] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:34:33.038313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e6472-b604-41d6-8bf7-c574e102e21b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:34:33.039633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f02e6472-b604-41d6-8bf7-c574e102e21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:33.050835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e6472-b604-41d6-8bf7-c574e102e21b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:34:33.113152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:34:33.147081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757100e-051c-4c79-b857-5f814733e24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:34:33.147756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757100e-051c-4c79-b857-5f814733e24f admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:33.147964 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757100e-051c-4c79-b857-5f814733e24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:33.148233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757100e-051c-4c79-b857-5f814733e24f admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:33.148869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757100e-051c-4c79-b857-5f814733e24f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:34:33.149202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757100e-051c-4c79-b857-5f814733e24f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:34:33.150242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: WARNING py.warnings [None req-c757100e-051c-4c79-b857-5f814733e24f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:34:33.150242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: warnings.warn(msg) Sep 28 08:34:33.150242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: Sep 28 08:34:33.151122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757100e-051c-4c79-b857-5f814733e24f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:34:33.163113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 733/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:34:33.171508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4eb0c3-1cfa-4669-9b8c-e10c0db81f02 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:34:33.181243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4eb0c3-1cfa-4669-9b8c-e10c0db81f02 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:34:33.243677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:34:33.277338 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c744b-1819-4bd9-a7a1-ce0afe639c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:34:33.277959 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c744b-1819-4bd9-a7a1-ce0afe639c7b admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:33.278227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c744b-1819-4bd9-a7a1-ce0afe639c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:33.278495 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c744b-1819-4bd9-a7a1-ce0afe639c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:33.279117 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c744b-1819-4bd9-a7a1-ce0afe639c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:34:33.280844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: WARNING py.warnings [None req-990c744b-1819-4bd9-a7a1-ce0afe639c7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:34:33.280844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: warnings.warn(msg) Sep 28 08:34:33.280844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: Sep 28 08:34:33.282071 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c744b-1819-4bd9-a7a1-ce0afe639c7b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:34:33.340424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 764/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:34:33 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:34:33.344879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757100e-051c-4c79-b857-5f814733e24f admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:34:33.346421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c757100e-051c-4c79-b857-5f814733e24f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:33.379643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757100e-051c-4c79-b857-5f814733e24f admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:34:33.476503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43ae5f-4a16-41d3-9d88-c7f9fb1fa6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:34:33.477129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43ae5f-4a16-41d3-9d88-c7f9fb1fa6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:33.477414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43ae5f-4a16-41d3-9d88-c7f9fb1fa6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:33.477687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43ae5f-4a16-41d3-9d88-c7f9fb1fa6e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:33.478337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43ae5f-4a16-41d3-9d88-c7f9fb1fa6e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:34:33.479349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: WARNING py.warnings [None req-ca43ae5f-4a16-41d3-9d88-c7f9fb1fa6e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:34:33.479349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: warnings.warn(msg) Sep 28 08:34:33.479349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: Sep 28 08:34:33.480560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43ae5f-4a16-41d3-9d88-c7f9fb1fa6e2 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:34:33.523417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 734/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:34:33 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:34:33.527954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c744b-1819-4bd9-a7a1-ce0afe639c7b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:34:33.529519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-990c744b-1819-4bd9-a7a1-ce0afe639c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:33.592666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c744b-1819-4bd9-a7a1-ce0afe639c7b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:34:33.681489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:34:33.692044 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df8058-47a0-4936-a06e-8c7aed7da2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:34:33.692645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df8058-47a0-4936-a06e-8c7aed7da2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:33.692918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df8058-47a0-4936-a06e-8c7aed7da2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:33.693211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df8058-47a0-4936-a06e-8c7aed7da2f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:33.693844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df8058-47a0-4936-a06e-8c7aed7da2f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:34:33.694839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: WARNING py.warnings [None req-67df8058-47a0-4936-a06e-8c7aed7da2f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:34:33.694839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: warnings.warn(msg) Sep 28 08:34:33.694839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: Sep 28 08:34:33.695708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df8058-47a0-4936-a06e-8c7aed7da2f5 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:34:33.708077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 765/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:34:33.712715 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43ae5f-4a16-41d3-9d88-c7f9fb1fa6e2 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:34:33.714261 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ca43ae5f-4a16-41d3-9d88-c7f9fb1fa6e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:33.726324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43ae5f-4a16-41d3-9d88-c7f9fb1fa6e2 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:34:33.751134 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:34:33.820958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ff1ed-f2cb-4d09-ae47-a2706f61b21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:34:33.821588 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ff1ed-f2cb-4d09-ae47-a2706f61b21a admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:33.821864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ff1ed-f2cb-4d09-ae47-a2706f61b21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:33.822130 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ff1ed-f2cb-4d09-ae47-a2706f61b21a admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:33.822765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ff1ed-f2cb-4d09-ae47-a2706f61b21a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:34:33.823764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: WARNING py.warnings [None req-219ff1ed-f2cb-4d09-ae47-a2706f61b21a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:34:33.823764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: warnings.warn(msg) Sep 28 08:34:33.823764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: Sep 28 08:34:33.824638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ff1ed-f2cb-4d09-ae47-a2706f61b21a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:34:33.837252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 735/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:34:33.841572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df8058-47a0-4936-a06e-8c7aed7da2f5 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:34:33.843097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-67df8058-47a0-4936-a06e-8c7aed7da2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:33.877682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df8058-47a0-4936-a06e-8c7aed7da2f5 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:34:33.972870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a7ca6-2a5d-4bc0-bd4e-1feffebd9c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:34:33.973579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a7ca6-2a5d-4bc0-bd4e-1feffebd9c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:33.973857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a7ca6-2a5d-4bc0-bd4e-1feffebd9c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:33.974128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a7ca6-2a5d-4bc0-bd4e-1feffebd9c29 admin admin] PATH_INFO: `/v3/projects/19a6548770c24e5a814ba23e50a31435/users/41f80237605b446295b4adbe8e05d61d/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:33.997426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a7ca6-2a5d-4bc0-bd4e-1feffebd9c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a6548770c24e5a814ba23e50a31435, user_id=41f80237605b446295b4adbe8e05d61d, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:34:33.998447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: WARNING py.warnings [None req-362a7ca6-2a5d-4bc0-bd4e-1feffebd9c29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:34:33.998447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: warnings.warn(msg) Sep 28 08:34:33.998447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: Sep 28 08:34:34.000373 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a7ca6-2a5d-4bc0-bd4e-1feffebd9c29 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:34:34.024271 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 766/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:34:33 2022] PUT /identity/v3/projects/19a6548770c24e5a814ba23e50a31435/users/41f80237605b446295b4adbe8e05d61d/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:34:34.037501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58064073-c982-49f4-8a68-06b39839209d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:34.037785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58064073-c982-49f4-8a68-06b39839209d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:34.038048 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58064073-c982-49f4-8a68-06b39839209d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:34.093595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-58064073-c982-49f4-8a68-06b39839209d None None] MFA Rules not processed for user `41f80237605b446295b4adbe8e05d61d`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:34:34.094784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-58064073-c982-49f4-8a68-06b39839209d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:34.173753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 736/1502] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:34:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:34:34.187577 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b63d43-543b-4880-af95-c02864debe0a None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:34.187860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b63d43-543b-4880-af95-c02864debe0a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:34.188132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b63d43-543b-4880-af95-c02864debe0a None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:34.189537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 767/1503] 10.222.0.52 () {54 vars in 963 bytes} [Wed Sep 28 08:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:34:34.194551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16327e-cb94-4545-90e8-fa6511e2f2cd None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:34.194843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16327e-cb94-4545-90e8-fa6511e2f2cd None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:34.195120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16327e-cb94-4545-90e8-fa6511e2f2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:34.227499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:34:34.270925 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ed16327e-cb94-4545-90e8-fa6511e2f2cd None None] MFA Rules not processed for user `5274740e5d7c4ab1ba9e9781815532c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:34:34.272652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ed16327e-cb94-4545-90e8-fa6511e2f2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:34.347389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 737/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Wed Sep 28 08:34:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:34:34.357652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b63d43-543b-4880-af95-c02864debe0a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:34:34.359281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f1b63d43-543b-4880-af95-c02864debe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:34.409325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:34:34.465688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b63d43-543b-4880-af95-c02864debe0a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:34:34.542326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747d117-4402-4d3d-89ad-7b9008b586bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:34:34.542943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747d117-4402-4d3d-89ad-7b9008b586bb service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:34.543270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747d117-4402-4d3d-89ad-7b9008b586bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:34.543538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747d117-4402-4d3d-89ad-7b9008b586bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:34.545216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3747d117-4402-4d3d-89ad-7b9008b586bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:34.648289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747d117-4402-4d3d-89ad-7b9008b586bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:34:34.649782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747d117-4402-4d3d-89ad-7b9008b586bb service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:34:34.653449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:34:34.669014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 768/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:34:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:34:34.715172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49917078-cd3a-4f92-8630-bc840d3ff5b3 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:34.715621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49917078-cd3a-4f92-8630-bc840d3ff5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:34.715716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49917078-cd3a-4f92-8630-bc840d3ff5b3 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:34.717009 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 738/1506] 10.222.0.52 () {52 vars in 830 bytes} [Wed Sep 28 08:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:34:34.720515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747d117-4402-4d3d-89ad-7b9008b586bb service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:34:34.721768 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3747d117-4402-4d3d-89ad-7b9008b586bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:34.826027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747d117-4402-4d3d-89ad-7b9008b586bb service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:34:34.923102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:34:34.923802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:34.923991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:34.924266 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] PATH_INFO: `/v3/users/41f80237605b446295b4adbe8e05d61d` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:34.938377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] RBAC: Authorizing `identity:delete_user(user_id=41f80237605b446295b4adbe8e05d61d)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:34:34.939424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: WARNING py.warnings [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:34:34.939424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: warnings.warn(msg) Sep 28 08:34:34.939424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: Sep 28 08:34:34.940638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:34:34.950652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:34:35.006321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41f80237605b446295b4adbe8e05d61d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:34:35.029069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41f80237605b446295b4adbe8e05d61d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:34:35.029563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41f80237605b446295b4adbe8e05d61d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:34:35.044398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41f80237605b446295b4adbe8e05d61d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:34:35.044948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41f80237605b446295b4adbe8e05d61d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:34:35.061083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 769/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:34:34 2022] DELETE /identity/v3/users/41f80237605b446295b4adbe8e05d61d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:34:35.065473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aff64e-c13b-4724-8066-902381ebe2aa None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:35.065762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aff64e-c13b-4724-8066-902381ebe2aa None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:35.066038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aff64e-c13b-4724-8066-902381ebe2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:35.124183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-59aff64e-c13b-4724-8066-902381ebe2aa None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:34:35.125327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-59aff64e-c13b-4724-8066-902381ebe2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:35.208276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:34:35.221617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 739/1508] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:34:35.231954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:34:35.233497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:35.335725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5617f1-43c5-4f31-94fd-1999edf0227d admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:34:35.392582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:34:35.414880 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d097eb4-a038-4be7-954e-0637b60a8dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:34:35.415494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d097eb4-a038-4be7-954e-0637b60a8dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:35.415766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d097eb4-a038-4be7-954e-0637b60a8dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:35.416037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d097eb4-a038-4be7-954e-0637b60a8dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:35.417650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7d097eb4-a038-4be7-954e-0637b60a8dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:35.528454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d097eb4-a038-4be7-954e-0637b60a8dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:34:35.529943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d097eb4-a038-4be7-954e-0637b60a8dd4 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:34:35.549428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 770/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:34:35.905404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aff64e-c13b-4724-8066-902381ebe2aa None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:34:35.906652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-59aff64e-c13b-4724-8066-902381ebe2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:34:35.918835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aff64e-c13b-4724-8066-902381ebe2aa None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:34:36.029978 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:34:36.030611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:34:36.030888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:34:36.031157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] PATH_INFO: `/v3/projects/19a6548770c24e5a814ba23e50a31435` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:34:36.042057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a6548770c24e5a814ba23e50a31435)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:34:36.043084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: WARNING py.warnings [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:34:36.043084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: warnings.warn(msg) Sep 28 08:34:36.043084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: Sep 28 08:34:36.044289 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:34:36.100949 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a6548770c24e5a814ba23e50a31435', 'request_id': 'req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:34:36.107501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a6548770c24e5a814ba23e50a31435', 'request_id': 'req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:34:36.117923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a6548770c24e5a814ba23e50a31435', 'request_id': 'req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:34:36.149714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] The token cache is being invalidate because project 19a6548770c24e5a814ba23e50a31435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:34:36.150014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:34:36.150447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e4552b3e-2da1-4212-8e4e-a83c58dd9f91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:34:36.151867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 740/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:34:35 2022] DELETE /identity/v3/projects/19a6548770c24e5a814ba23e50a31435 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:34:45.547030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=62 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:34:46.147531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:35:59.349827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67474f36-1930-46ba-a620-a01067c0fa18 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:35:59.350402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67474f36-1930-46ba-a620-a01067c0fa18 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:35:59.350402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67474f36-1930-46ba-a620-a01067c0fa18 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:35:59.351758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 771/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 08:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:35:59.356749 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9507899-56de-49c9-8485-982bd37dc065 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:35:59.357201 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9507899-56de-49c9-8485-982bd37dc065 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:35:59.357343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9507899-56de-49c9-8485-982bd37dc065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:35:59.452904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-b9507899-56de-49c9-8485-982bd37dc065 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:35:59.454164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b9507899-56de-49c9-8485-982bd37dc065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:35:59.560584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 741/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 08:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:35:59.716819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67474f36-1930-46ba-a620-a01067c0fa18 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:35:59.718971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-67474f36-1930-46ba-a620-a01067c0fa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:35:59.840168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67474f36-1930-46ba-a620-a01067c0fa18 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:35:59.845567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73096d2-323d-4220-bbe7-f49eef1bf775 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:35:59.846171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73096d2-323d-4220-bbe7-f49eef1bf775 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:35:59.846472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73096d2-323d-4220-bbe7-f49eef1bf775 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:35:59.846727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73096d2-323d-4220-bbe7-f49eef1bf775 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:35:59.854875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73096d2-323d-4220-bbe7-f49eef1bf775 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:35:59.856476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73096d2-323d-4220-bbe7-f49eef1bf775 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:35:59.865826 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 772/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:35:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:36:00.956487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab038b2-4219-4f5f-bd2c-5c4d36ee14d4 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:00.957020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab038b2-4219-4f5f-bd2c-5c4d36ee14d4 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:00.957020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab038b2-4219-4f5f-bd2c-5c4d36ee14d4 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:00.958412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 742/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 08:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:36:00.966087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5c1c6-b26b-4d33-873d-47e21552a95e None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:00.966569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5c1c6-b26b-4d33-873d-47e21552a95e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:00.966671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5c1c6-b26b-4d33-873d-47e21552a95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:00.985921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-16b5c1c6-b26b-4d33-873d-47e21552a95e None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:36:00.987138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-16b5c1c6-b26b-4d33-873d-47e21552a95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:36:00.995116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 773/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 08:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:36:01.151785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab038b2-4219-4f5f-bd2c-5c4d36ee14d4 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:36:01.166537 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab038b2-4219-4f5f-bd2c-5c4d36ee14d4 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:36:01.171966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71070d45-c34a-4ca8-be20-105c91117c6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:36:01.172560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71070d45-c34a-4ca8-be20-105c91117c6e service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:01.172836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71070d45-c34a-4ca8-be20-105c91117c6e service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:01.173132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71070d45-c34a-4ca8-be20-105c91117c6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:01.181709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71070d45-c34a-4ca8-be20-105c91117c6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:36:01.183282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71070d45-c34a-4ca8-be20-105c91117c6e service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:36:01.192532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 743/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:36:01.307996 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dbb6b-3a0d-46cc-b11d-f7e4859c8066 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:01.308283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dbb6b-3a0d-46cc-b11d-f7e4859c8066 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:01.308559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dbb6b-3a0d-46cc-b11d-f7e4859c8066 None None] PATH_INFO: `/v3` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:01.309943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 774/1517] 10.222.0.26 () {54 vars in 925 bytes} [Wed Sep 28 08:36:01 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:36:01.316108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838a5a1-5cd8-46f3-b41f-b96a69eabd7c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:01.316396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838a5a1-5cd8-46f3-b41f-b96a69eabd7c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:01.316672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838a5a1-5cd8-46f3-b41f-b96a69eabd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:01.377304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-9838a5a1-5cd8-46f3-b41f-b96a69eabd7c None None] MFA Rules not processed for user `6b40ee263df74e58bf3f958c01eb742f`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:36:01.378527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9838a5a1-5cd8-46f3-b41f-b96a69eabd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:36:01.396333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:36:01.451765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 744/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Sep 28 08:36:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:36:10.108166 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5d4a4-8c9c-44ad-9ece-3b191552bf01 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:10.108665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5d4a4-8c9c-44ad-9ece-3b191552bf01 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:10.108795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5d4a4-8c9c-44ad-9ece-3b191552bf01 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:10.110054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 775/1519] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 08:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:36:10.115141 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d687c5-dc26-45bf-9ca1-0f146a29e011 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:10.115602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d687c5-dc26-45bf-9ca1-0f146a29e011 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:10.115748 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d687c5-dc26-45bf-9ca1-0f146a29e011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:10.134912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-93d687c5-dc26-45bf-9ca1-0f146a29e011 None None] MFA Rules not processed for user `6b40ee263df74e58bf3f958c01eb742f`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:36:10.136122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-93d687c5-dc26-45bf-9ca1-0f146a29e011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:36:10.145166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 745/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 08:36:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:36:10.153180 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5d4a4-8c9c-44ad-9ece-3b191552bf01 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:36:10.168286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5d4a4-8c9c-44ad-9ece-3b191552bf01 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:36:10.173388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088eea3-a657-4244-94c1-2fe75c985758 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:36:10.174000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088eea3-a657-4244-94c1-2fe75c985758 service placement] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:10.174283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088eea3-a657-4244-94c1-2fe75c985758 service placement] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:10.174558 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088eea3-a657-4244-94c1-2fe75c985758 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:10.183435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088eea3-a657-4244-94c1-2fe75c985758 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:36:10.185033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088eea3-a657-4244-94c1-2fe75c985758 service placement] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:36:10.194652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 776/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:36:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:36:12.694801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802b0cf-cc13-479f-8a92-d25320aa812e None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:12.695386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802b0cf-cc13-479f-8a92-d25320aa812e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:12.695386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802b0cf-cc13-479f-8a92-d25320aa812e None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:12.697285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 746/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 08:36:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:36:12.703511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ed0a6-6b17-4498-8a1e-79d00b81bc1b None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:12.703965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ed0a6-6b17-4498-8a1e-79d00b81bc1b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:12.704071 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ed0a6-6b17-4498-8a1e-79d00b81bc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:12.723830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-602ed0a6-6b17-4498-8a1e-79d00b81bc1b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:36:12.725016 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-602ed0a6-6b17-4498-8a1e-79d00b81bc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:36:12.733259 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 777/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 08:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:36:12.909934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802b0cf-cc13-479f-8a92-d25320aa812e None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:36:12.924825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802b0cf-cc13-479f-8a92-d25320aa812e None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:36:12.930962 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b620461-b576-43bc-852f-d9af9d93902b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:36:12.931561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b620461-b576-43bc-852f-d9af9d93902b service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:12.931840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b620461-b576-43bc-852f-d9af9d93902b service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:12.932113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b620461-b576-43bc-852f-d9af9d93902b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:12.940183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b620461-b576-43bc-852f-d9af9d93902b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:36:12.941794 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b620461-b576-43bc-852f-d9af9d93902b service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:36:12.951101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 747/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:36:14.046687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe29c48-1588-488b-b6d7-c46196bca07a None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:14.047323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe29c48-1588-488b-b6d7-c46196bca07a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:14.047323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe29c48-1588-488b-b6d7-c46196bca07a None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:14.048565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 778/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 08:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:36:14.054923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57209df6-f198-457a-9ed6-311b2733ff1b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:14.055422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57209df6-f198-457a-9ed6-311b2733ff1b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:14.055525 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57209df6-f198-457a-9ed6-311b2733ff1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:14.074434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-57209df6-f198-457a-9ed6-311b2733ff1b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:36:14.075614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-57209df6-f198-457a-9ed6-311b2733ff1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:36:14.083254 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 748/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 08:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:36:14.238873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe29c48-1588-488b-b6d7-c46196bca07a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:36:14.253630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe29c48-1588-488b-b6d7-c46196bca07a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:36:14.259148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf02a41-a134-4240-a8d1-159f87323c9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:36:14.259747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf02a41-a134-4240-a8d1-159f87323c9c service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:14.260030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf02a41-a134-4240-a8d1-159f87323c9c service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:14.260302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf02a41-a134-4240-a8d1-159f87323c9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:14.268262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf02a41-a134-4240-a8d1-159f87323c9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:36:14.269849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf02a41-a134-4240-a8d1-159f87323c9c service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:36:14.279374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 779/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:36:18.253196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca2462-be02-445a-971e-8912b09475c3 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:18.254124 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca2462-be02-445a-971e-8912b09475c3 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:18.254124 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca2462-be02-445a-971e-8912b09475c3 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:18.255707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 749/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:36:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:36:18.261171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f701cad-3813-4ec3-bdd4-c56688cedcfd None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:18.261629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f701cad-3813-4ec3-bdd4-c56688cedcfd None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:18.261756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f701cad-3813-4ec3-bdd4-c56688cedcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:18.281126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-8f701cad-3813-4ec3-bdd4-c56688cedcfd None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:36:18.282337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8f701cad-3813-4ec3-bdd4-c56688cedcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:36:18.291251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 780/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:36:18.582756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca2462-be02-445a-971e-8912b09475c3 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:36:18.584509 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e3ca2462-be02-445a-971e-8912b09475c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:36:18.673738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca2462-be02-445a-971e-8912b09475c3 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:36:18.677539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cc66f-1470-4140-8a9b-d7c004f3243f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:36:18.678168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cc66f-1470-4140-8a9b-d7c004f3243f service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:18.678474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cc66f-1470-4140-8a9b-d7c004f3243f service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:18.678748 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cc66f-1470-4140-8a9b-d7c004f3243f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:18.689366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083cc66f-1470-4140-8a9b-d7c004f3243f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:36:18.690848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083cc66f-1470-4140-8a9b-d7c004f3243f service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:36:18.702125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 750/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:36:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:36:20.921686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fddd9-ca65-4f85-90dd-46b83701eb9c None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:20.922243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fddd9-ca65-4f85-90dd-46b83701eb9c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:20.922243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fddd9-ca65-4f85-90dd-46b83701eb9c None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:20.923489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 781/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:36:20.930048 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95cb06-f699-4909-893d-54f2d63b325f None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:20.930499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95cb06-f699-4909-893d-54f2d63b325f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:20.930599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95cb06-f699-4909-893d-54f2d63b325f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:20.950973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-9e95cb06-f699-4909-893d-54f2d63b325f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:36:20.952198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9e95cb06-f699-4909-893d-54f2d63b325f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:36:20.960728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 751/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:36:21.256390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fddd9-ca65-4f85-90dd-46b83701eb9c None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:36:21.267168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fddd9-ca65-4f85-90dd-46b83701eb9c None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:36:21.270692 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ea37e-230d-41a4-8b01-aebb43a1149e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:36:21.271289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea37e-230d-41a4-8b01-aebb43a1149e service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:21.271562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea37e-230d-41a4-8b01-aebb43a1149e service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:21.271832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea37e-230d-41a4-8b01-aebb43a1149e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:21.280952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ea37e-230d-41a4-8b01-aebb43a1149e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:36:21.282387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ea37e-230d-41a4-8b01-aebb43a1149e service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:36:21.293247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 782/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:36:25.012431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510db057-8595-40f9-bbfb-20ff4c6a1013 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:25.012971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510db057-8595-40f9-bbfb-20ff4c6a1013 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:25.013292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510db057-8595-40f9-bbfb-20ff4c6a1013 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:25.014654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 752/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:36:25.020271 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59c27a-1fec-4db9-a181-fad8e37537af None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:25.020751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59c27a-1fec-4db9-a181-fad8e37537af None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:25.020848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59c27a-1fec-4db9-a181-fad8e37537af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:25.040692 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-3f59c27a-1fec-4db9-a181-fad8e37537af None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:36:25.042427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3f59c27a-1fec-4db9-a181-fad8e37537af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:36:25.049765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 783/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:36:25.364501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510db057-8595-40f9-bbfb-20ff4c6a1013 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:36:25.366404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-510db057-8595-40f9-bbfb-20ff4c6a1013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:36:25.437980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:36:25.451818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510db057-8595-40f9-bbfb-20ff4c6a1013 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:36:25.455422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efd7c7-9480-472e-ac5c-30857afd5e19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:36:25.456022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efd7c7-9480-472e-ac5c-30857afd5e19 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:25.456302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efd7c7-9480-472e-ac5c-30857afd5e19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:25.456565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efd7c7-9480-472e-ac5c-30857afd5e19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:25.467059 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efd7c7-9480-472e-ac5c-30857afd5e19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:36:25.468504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efd7c7-9480-472e-ac5c-30857afd5e19 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:36:25.478727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 753/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:36:27.664123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ab454-bf6c-4dc8-be60-3397a2c67e34 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:27.664748 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ab454-bf6c-4dc8-be60-3397a2c67e34 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:27.664748 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ab454-bf6c-4dc8-be60-3397a2c67e34 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:27.665936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 784/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 08:36:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:36:27.671300 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc49a81-c8bb-47ec-bfd9-ed375487206b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:27.671749 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc49a81-c8bb-47ec-bfd9-ed375487206b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:27.671853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc49a81-c8bb-47ec-bfd9-ed375487206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:27.692080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-edc49a81-c8bb-47ec-bfd9-ed375487206b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:36:27.693323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-edc49a81-c8bb-47ec-bfd9-ed375487206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:36:27.701082 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 754/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 08:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:36:28.003101 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ab454-bf6c-4dc8-be60-3397a2c67e34 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:36:28.004811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-740ab454-bf6c-4dc8-be60-3397a2c67e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:36:28.012786 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=84 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:36:28.019197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ab454-bf6c-4dc8-be60-3397a2c67e34 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:36:28.022721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2d06a-74c3-4155-9b6d-70ae3579b2c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:36:28.024207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2d06a-74c3-4155-9b6d-70ae3579b2c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:36:28.024481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2d06a-74c3-4155-9b6d-70ae3579b2c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:36:28.024749 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2d06a-74c3-4155-9b6d-70ae3579b2c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:36:28.033556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2d06a-74c3-4155-9b6d-70ae3579b2c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:36:28.034968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2d06a-74c3-4155-9b6d-70ae3579b2c9 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:36:28.044439 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 785/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:36:37.686204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=11 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:36:38.040338 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:51.950597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545f7ce-df7f-4e44-baf7-11edb3122f15 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:51.952095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545f7ce-df7f-4e44-baf7-11edb3122f15 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:51.952605 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545f7ce-df7f-4e44-baf7-11edb3122f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:51.980016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-7545f7ce-df7f-4e44-baf7-11edb3122f15 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:51.981664 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7545f7ce-df7f-4e44-baf7-11edb3122f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:51.994828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 755/1540] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.003887 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b17284-3277-4a2e-b5f6-5983847ae813 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.004447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b17284-3277-4a2e-b5f6-5983847ae813 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.004447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b17284-3277-4a2e-b5f6-5983847ae813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.020630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0fc30-a2bb-4f33-bc2b-ac182ac10209 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.021347 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0fc30-a2bb-4f33-bc2b-ac182ac10209 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.021863 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0fc30-a2bb-4f33-bc2b-ac182ac10209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.038401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-d6b17284-3277-4a2e-b5f6-5983847ae813 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.039770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d6b17284-3277-4a2e-b5f6-5983847ae813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.055165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 786/1541] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.058124 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-fec0fc30-a2bb-4f33-bc2b-ac182ac10209 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.059347 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-fec0fc30-a2bb-4f33-bc2b-ac182ac10209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.064330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b17284-3277-4a2e-b5f6-5983847ae813 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.074115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 756/1542] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.079315 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b17284-3277-4a2e-b5f6-5983847ae813 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.085410 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2c4be-2b70-41ff-940f-f6f746865593 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.086017 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2c4be-2b70-41ff-940f-f6f746865593 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.086495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2c4be-2b70-41ff-940f-f6f746865593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.087711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce36011-cabf-42b0-8771-543345322c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.088406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce36011-cabf-42b0-8771-543345322c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.088755 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce36011-cabf-42b0-8771-543345322c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.089121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce36011-cabf-42b0-8771-543345322c02 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.089981 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce36011-cabf-42b0-8771-543345322c02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:37:52.090371 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce36011-cabf-42b0-8771-543345322c02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.093115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: WARNING py.warnings [None req-3ce36011-cabf-42b0-8771-543345322c02 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:37:52.093115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: warnings.warn(msg) Sep 28 08:37:52.093115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: Sep 28 08:37:52.093115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce36011-cabf-42b0-8771-543345322c02 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.113888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 787/1543] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:37:52.120663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cff30-63cd-4890-9e03-920206e22797 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.120663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cff30-63cd-4890-9e03-920206e22797 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.120663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cff30-63cd-4890-9e03-920206e22797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.128210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-7ac2c4be-2b70-41ff-940f-f6f746865593 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.129795 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7ac2c4be-2b70-41ff-940f-f6f746865593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.144630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-ec8cff30-63cd-4890-9e03-920206e22797 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.148210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 757/1544] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:37:52.149307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2c4be-2b70-41ff-940f-f6f746865593 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.150829 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ec8cff30-63cd-4890-9e03-920206e22797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.167893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2c4be-2b70-41ff-940f-f6f746865593 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.185108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04d2b0-886e-486f-aa07-00d07c33fa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.186046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 788/1545] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:37:52.186389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04d2b0-886e-486f-aa07-00d07c33fa75 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.186915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04d2b0-886e-486f-aa07-00d07c33fa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.187265 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8cff30-63cd-4890-9e03-920206e22797 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.187621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04d2b0-886e-486f-aa07-00d07c33fa75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.188506 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04d2b0-886e-486f-aa07-00d07c33fa75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.189944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: WARNING py.warnings [None req-8c04d2b0-886e-486f-aa07-00d07c33fa75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:37:52.189944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: warnings.warn(msg) Sep 28 08:37:52.189944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: Sep 28 08:37:52.191541 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04d2b0-886e-486f-aa07-00d07c33fa75 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.206629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8cff30-63cd-4890-9e03-920206e22797 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.216685 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c1f7a-d474-4c1a-8311-eacce584eadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.218372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c1f7a-d474-4c1a-8311-eacce584eadd admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.219352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c1f7a-d474-4c1a-8311-eacce584eadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.221176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c1f7a-d474-4c1a-8311-eacce584eadd admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.222058 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c1f7a-d474-4c1a-8311-eacce584eadd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:37:52.222553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c1f7a-d474-4c1a-8311-eacce584eadd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.225853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c1f7a-d474-4c1a-8311-eacce584eadd admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.242572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 789/1546] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:37:52.244707 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822c999-8edd-471e-8611-917b10e9d262 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.245201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822c999-8edd-471e-8611-917b10e9d262 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.245660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822c999-8edd-471e-8611-917b10e9d262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.254852 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 758/1547] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:52.256603 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8622dd6-e8dd-4282-9dc3-dd22e76cba7c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.256927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8622dd6-e8dd-4282-9dc3-dd22e76cba7c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.257281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8622dd6-e8dd-4282-9dc3-dd22e76cba7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.272747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5822c999-8edd-471e-8611-917b10e9d262 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.273979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5822c999-8edd-471e-8611-917b10e9d262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.279074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-f8622dd6-e8dd-4282-9dc3-dd22e76cba7c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.280887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f8622dd6-e8dd-4282-9dc3-dd22e76cba7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.336522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 790/1548] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.338060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c63be-521a-4ffb-b91a-67ed1283a062 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.338512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c63be-521a-4ffb-b91a-67ed1283a062 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.338741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c63be-521a-4ffb-b91a-67ed1283a062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.344928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 759/1549] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.346349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95a535-698e-479d-8e0d-025ddfa7eaf9 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.346961 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95a535-698e-479d-8e0d-025ddfa7eaf9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.347091 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95a535-698e-479d-8e0d-025ddfa7eaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.363759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-652c63be-521a-4ffb-b91a-67ed1283a062 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.364963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-652c63be-521a-4ffb-b91a-67ed1283a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.366622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-bc95a535-698e-479d-8e0d-025ddfa7eaf9 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.367796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-bc95a535-698e-479d-8e0d-025ddfa7eaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.372873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 791/1550] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.373678 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c63be-521a-4ffb-b91a-67ed1283a062 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.381142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 760/1551] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.381142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95a535-698e-479d-8e0d-025ddfa7eaf9 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.383406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c63be-521a-4ffb-b91a-67ed1283a062 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.387029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95a535-698e-479d-8e0d-025ddfa7eaf9 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.388180 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a0c2d-9a27-447d-90cf-b090e944f551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.388827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a0c2d-9a27-447d-90cf-b090e944f551 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.389671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a0c2d-9a27-447d-90cf-b090e944f551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.389951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a0c2d-9a27-447d-90cf-b090e944f551 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.390595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a0c2d-9a27-447d-90cf-b090e944f551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.392400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6136a0f-e95c-4276-b107-0cea2cc31149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.392742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a0c2d-9a27-447d-90cf-b090e944f551 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.393001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6136a0f-e95c-4276-b107-0cea2cc31149 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.393392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6136a0f-e95c-4276-b107-0cea2cc31149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.393673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6136a0f-e95c-4276-b107-0cea2cc31149 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.394303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6136a0f-e95c-4276-b107-0cea2cc31149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.396199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6136a0f-e95c-4276-b107-0cea2cc31149 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.419926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 761/1552] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:52.423222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58790640-cda8-4dfa-b2be-ac262f76354c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.423222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58790640-cda8-4dfa-b2be-ac262f76354c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.423222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58790640-cda8-4dfa-b2be-ac262f76354c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.428918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 792/1553] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:52.428918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188d07f-402e-406d-bd0e-6af677e55030 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.428918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188d07f-402e-406d-bd0e-6af677e55030 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.428918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188d07f-402e-406d-bd0e-6af677e55030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.441649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-58790640-cda8-4dfa-b2be-ac262f76354c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.442871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-58790640-cda8-4dfa-b2be-ac262f76354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.447647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b188d07f-402e-406d-bd0e-6af677e55030 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.448833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b188d07f-402e-406d-bd0e-6af677e55030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.505182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 762/1554] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.505507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1241ba-3de2-427a-9e6b-39ba507793c1 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.505763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1241ba-3de2-427a-9e6b-39ba507793c1 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.506038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1241ba-3de2-427a-9e6b-39ba507793c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.519218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 793/1555] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.520856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852daeec-089d-48c3-be80-69a8ca794856 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.521197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852daeec-089d-48c3-be80-69a8ca794856 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.521447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852daeec-089d-48c3-be80-69a8ca794856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.535843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ec1241ba-3de2-427a-9e6b-39ba507793c1 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.537155 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ec1241ba-3de2-427a-9e6b-39ba507793c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.545225 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-852daeec-089d-48c3-be80-69a8ca794856 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.545976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 763/1556] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.546600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-852daeec-089d-48c3-be80-69a8ca794856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.549494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1241ba-3de2-427a-9e6b-39ba507793c1 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.556241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 794/1557] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.558504 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf7d50-8a25-4315-9c96-287417910914 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.558993 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf7d50-8a25-4315-9c96-287417910914 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.559464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf7d50-8a25-4315-9c96-287417910914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.560856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1241ba-3de2-427a-9e6b-39ba507793c1 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.566737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeff6cc-af37-4e2d-819e-b6a6e1420cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.567354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff6cc-af37-4e2d-819e-b6a6e1420cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.567642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff6cc-af37-4e2d-819e-b6a6e1420cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.567912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeff6cc-af37-4e2d-819e-b6a6e1420cb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.568592 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeff6cc-af37-4e2d-819e-b6a6e1420cb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:37:52.568896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeff6cc-af37-4e2d-819e-b6a6e1420cb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.570545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeff6cc-af37-4e2d-819e-b6a6e1420cb6 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.583626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 764/1558] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:37:52.584400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-51bf7d50-8a25-4315-9c96-287417910914 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.584589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeff6cc-af37-4e2d-819e-b6a6e1420cb6 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.585862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-51bf7d50-8a25-4315-9c96-287417910914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.595506 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeff6cc-af37-4e2d-819e-b6a6e1420cb6 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.597974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 795/1559] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.598870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf7d50-8a25-4315-9c96-287417910914 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.600377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947a03c-e75d-4338-bdde-56ba9acbc81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.601001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947a03c-e75d-4338-bdde-56ba9acbc81b admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.601315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947a03c-e75d-4338-bdde-56ba9acbc81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.601591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947a03c-e75d-4338-bdde-56ba9acbc81b admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.602238 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947a03c-e75d-4338-bdde-56ba9acbc81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.603851 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947a03c-e75d-4338-bdde-56ba9acbc81b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.605372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:52.609509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf7d50-8a25-4315-9c96-287417910914 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.615326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c510c-3719-469b-8c69-5ef033921e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.616034 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c510c-3719-469b-8c69-5ef033921e94 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.616316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c510c-3719-469b-8c69-5ef033921e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.616621 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c510c-3719-469b-8c69-5ef033921e94 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.617588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c510c-3719-469b-8c69-5ef033921e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.618729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: WARNING py.warnings [None req-497c510c-3719-469b-8c69-5ef033921e94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:37:52.618729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: warnings.warn(msg) Sep 28 08:37:52.618729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: Sep 28 08:37:52.619952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c510c-3719-469b-8c69-5ef033921e94 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.620680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 765/1560] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:52.622068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6fb99-6676-46e9-aeb6-8a1fcc31edd6 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.622353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6fb99-6676-46e9-aeb6-8a1fcc31edd6 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.622793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6fb99-6676-46e9-aeb6-8a1fcc31edd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.646182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-c8c6fb99-6676-46e9-aeb6-8a1fcc31edd6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.647745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c8c6fb99-6676-46e9-aeb6-8a1fcc31edd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.649958 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 796/1561] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/users => generated 293 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:52.652300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d518f93-17c3-4946-9abf-551de0e12714 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.652902 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d518f93-17c3-4946-9abf-551de0e12714 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.653444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d518f93-17c3-4946-9abf-551de0e12714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.659410 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 766/1562] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.662023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6fb99-6676-46e9-aeb6-8a1fcc31edd6 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.668904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:52.672980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6fb99-6676-46e9-aeb6-8a1fcc31edd6 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.677628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c4e5b-e980-4cf5-9f6b-f48ab00825b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.678239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c4e5b-e980-4cf5-9f6b-f48ab00825b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.678520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c4e5b-e980-4cf5-9f6b-f48ab00825b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.678778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c4e5b-e980-4cf5-9f6b-f48ab00825b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.679420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c4e5b-e980-4cf5-9f6b-f48ab00825b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:37:52.679733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c4e5b-e980-4cf5-9f6b-f48ab00825b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.681387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c4e5b-e980-4cf5-9f6b-f48ab00825b7 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.688556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-9d518f93-17c3-4946-9abf-551de0e12714 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:52.690748 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9d518f93-17c3-4946-9abf-551de0e12714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:52.693943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 767/1563] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:37:52.694770 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c4e5b-e980-4cf5-9f6b-f48ab00825b7 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.704376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 797/1564] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:52.705326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d518f93-17c3-4946-9abf-551de0e12714 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.706781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c4e5b-e980-4cf5-9f6b-f48ab00825b7 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.711651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef16afc-caea-4b09-8849-715e3ea1c36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.712257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef16afc-caea-4b09-8849-715e3ea1c36e admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.712539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef16afc-caea-4b09-8849-715e3ea1c36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.712808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef16afc-caea-4b09-8849-715e3ea1c36e admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.713558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef16afc-caea-4b09-8849-715e3ea1c36e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:37:52.713855 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef16afc-caea-4b09-8849-715e3ea1c36e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.715190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d518f93-17c3-4946-9abf-551de0e12714 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.715447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef16afc-caea-4b09-8849-715e3ea1c36e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.719906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d4067-1b98-45cf-9602-46843f5965df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.720426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d4067-1b98-45cf-9602-46843f5965df admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.720704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d4067-1b98-45cf-9602-46843f5965df admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.720977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d4067-1b98-45cf-9602-46843f5965df admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.721644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d4067-1b98-45cf-9602-46843f5965df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.724266 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d4067-1b98-45cf-9602-46843f5965df admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.728429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 768/1565] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:37:52.729082 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef16afc-caea-4b09-8849-715e3ea1c36e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.739376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef16afc-caea-4b09-8849-715e3ea1c36e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.744189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b4887-ff64-46d7-b107-7ae95886e652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.744788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b4887-ff64-46d7-b107-7ae95886e652 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.745060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b4887-ff64-46d7-b107-7ae95886e652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.745369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b4887-ff64-46d7-b107-7ae95886e652 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.746003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b4887-ff64-46d7-b107-7ae95886e652 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:37:52.746301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b4887-ff64-46d7-b107-7ae95886e652 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.747911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b4887-ff64-46d7-b107-7ae95886e652 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.764849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 798/1566] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:52.764849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d4067-1b98-45cf-9602-46843f5965df admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.765154 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 769/1567] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:37:52.765154 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b4887-ff64-46d7-b107-7ae95886e652 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.775783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b4887-ff64-46d7-b107-7ae95886e652 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.834147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d4067-1b98-45cf-9602-46843f5965df admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.838962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672546b-2072-47a7-9d5d-2ab267f91f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.839585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672546b-2072-47a7-9d5d-2ab267f91f57 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.839784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c465d95-fa08-4ef3-ac6d-6cb8e596261a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.839972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672546b-2072-47a7-9d5d-2ab267f91f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.840114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672546b-2072-47a7-9d5d-2ab267f91f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.840756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c465d95-fa08-4ef3-ac6d-6cb8e596261a admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.840902 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672546b-2072-47a7-9d5d-2ab267f91f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.841136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c465d95-fa08-4ef3-ac6d-6cb8e596261a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.841479 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c465d95-fa08-4ef3-ac6d-6cb8e596261a admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.842206 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c465d95-fa08-4ef3-ac6d-6cb8e596261a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.842458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672546b-2072-47a7-9d5d-2ab267f91f57 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.844091 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c465d95-fa08-4ef3-ac6d-6cb8e596261a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.856634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 799/1568] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:52.856634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672546b-2072-47a7-9d5d-2ab267f91f57 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.856956 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 770/1569] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:52.856956 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c465d95-fa08-4ef3-ac6d-6cb8e596261a admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.867411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c465d95-fa08-4ef3-ac6d-6cb8e596261a admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.868120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672546b-2072-47a7-9d5d-2ab267f91f57 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.872725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d5547-b736-4782-82c3-bd5eb2b93f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.874093 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0686230-92f1-4605-b72b-72ee06be6c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.874267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d5547-b736-4782-82c3-bd5eb2b93f79 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.874369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d5547-b736-4782-82c3-bd5eb2b93f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.874642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d5547-b736-4782-82c3-bd5eb2b93f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.874738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0686230-92f1-4605-b72b-72ee06be6c1f admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.874983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0686230-92f1-4605-b72b-72ee06be6c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.875247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0686230-92f1-4605-b72b-72ee06be6c1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.875363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d5547-b736-4782-82c3-bd5eb2b93f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.875908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0686230-92f1-4605-b72b-72ee06be6c1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:37:52.876208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0686230-92f1-4605-b72b-72ee06be6c1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.877230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d5547-b736-4782-82c3-bd5eb2b93f79 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.877823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0686230-92f1-4605-b72b-72ee06be6c1f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.891834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 800/1570] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:37:52.893534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0686230-92f1-4605-b72b-72ee06be6c1f admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.904947 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0686230-92f1-4605-b72b-72ee06be6c1f admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.909689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb38ce-ab49-4fdd-882c-b1fe0f85a1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.910306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb38ce-ab49-4fdd-882c-b1fe0f85a1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.910491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 771/1571] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:52.910626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb38ce-ab49-4fdd-882c-b1fe0f85a1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.910852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb38ce-ab49-4fdd-882c-b1fe0f85a1f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.911243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d5547-b736-4782-82c3-bd5eb2b93f79 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.911507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb38ce-ab49-4fdd-882c-b1fe0f85a1f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:37:52.911871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb38ce-ab49-4fdd-882c-b1fe0f85a1f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.913857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb38ce-ab49-4fdd-882c-b1fe0f85a1f8 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:52.926869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 801/1572] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:37:52.927671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb38ce-ab49-4fdd-882c-b1fe0f85a1f8 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:52.947278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb38ce-ab49-4fdd-882c-b1fe0f85a1f8 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.980763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d5547-b736-4782-82c3-bd5eb2b93f79 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:52.988742 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63699ac-c524-4346-8afe-7134274ce5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:52.989396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63699ac-c524-4346-8afe-7134274ce5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:52.989674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63699ac-c524-4346-8afe-7134274ce5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:52.989946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63699ac-c524-4346-8afe-7134274ce5a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:52.990920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63699ac-c524-4346-8afe-7134274ce5a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:52.992513 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63699ac-c524-4346-8afe-7134274ce5a1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.017126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fd37c-d663-459e-9cf1-95ab7254d0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.018004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fd37c-d663-459e-9cf1-95ab7254d0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.018288 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fd37c-d663-459e-9cf1-95ab7254d0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.018565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3fd37c-d663-459e-9cf1-95ab7254d0d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.019209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fd37c-d663-459e-9cf1-95ab7254d0d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.021546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3fd37c-d663-459e-9cf1-95ab7254d0d6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.027867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 772/1573] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:53.027867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63699ac-c524-4346-8afe-7134274ce5a1 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.035956 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:53.046571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 802/1574] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:37:52 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:53.048384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fd37c-d663-459e-9cf1-95ab7254d0d6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.059892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3fd37c-d663-459e-9cf1-95ab7254d0d6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.091756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:53.095916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63699ac-c524-4346-8afe-7134274ce5a1 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.101246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1fdf9-79e4-4596-9ab1-7d29b27758ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.101864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1fdf9-79e4-4596-9ab1-7d29b27758ad admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.102276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1fdf9-79e4-4596-9ab1-7d29b27758ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.102578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1fdf9-79e4-4596-9ab1-7d29b27758ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.103288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1fdf9-79e4-4596-9ab1-7d29b27758ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.105293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1fdf9-79e4-4596-9ab1-7d29b27758ad admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.122602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58681734-1f25-4c48-b0eb-92215dd2ccd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.122963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58681734-1f25-4c48-b0eb-92215dd2ccd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.123326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58681734-1f25-4c48-b0eb-92215dd2ccd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.123561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58681734-1f25-4c48-b0eb-92215dd2ccd6 admin admin] PATH_INFO: `/v3/projects/9a008845495a4667aded1f8a6f191b3f/users/1b3bcd7fc8114058884eb8ab09aaa154/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.135324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 773/1575] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:37:53 2022] POST /identity/v3/projects => generated 391 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:53.136339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1fdf9-79e4-4596-9ab1-7d29b27758ad admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.146673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58681734-1f25-4c48-b0eb-92215dd2ccd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a008845495a4667aded1f8a6f191b3f, user_id=1b3bcd7fc8114058884eb8ab09aaa154, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.149414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58681734-1f25-4c48-b0eb-92215dd2ccd6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.166062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 803/1576] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:37:53 2022] PUT /identity/v3/projects/9a008845495a4667aded1f8a6f191b3f/users/1b3bcd7fc8114058884eb8ab09aaa154/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:53.167030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58681734-1f25-4c48-b0eb-92215dd2ccd6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.211555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1fdf9-79e4-4596-9ab1-7d29b27758ad admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.236943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58681734-1f25-4c48-b0eb-92215dd2ccd6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.241734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4eb17-14c3-4a88-a20c-ac97419a437c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.242361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4eb17-14c3-4a88-a20c-ac97419a437c admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.242654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4eb17-14c3-4a88-a20c-ac97419a437c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.242900 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4eb17-14c3-4a88-a20c-ac97419a437c admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.243538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4eb17-14c3-4a88-a20c-ac97419a437c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.245595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4eb17-14c3-4a88-a20c-ac97419a437c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.272504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e4092-81da-405c-93ef-df5fefd83663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.273027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e4092-81da-405c-93ef-df5fefd83663 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.273338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e4092-81da-405c-93ef-df5fefd83663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.273606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e4092-81da-405c-93ef-df5fefd83663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.274247 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e4092-81da-405c-93ef-df5fefd83663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.278192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 804/1577] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:37:53 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:53.278365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e4092-81da-405c-93ef-df5fefd83663 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.278469 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4eb17-14c3-4a88-a20c-ac97419a437c admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.291456 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 774/1578] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:53.291456 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e4092-81da-405c-93ef-df5fefd83663 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.300318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e4092-81da-405c-93ef-df5fefd83663 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.337907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:53.346063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4eb17-14c3-4a88-a20c-ac97419a437c admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.351169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05a830-e3d4-4159-a38c-675c21e47d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.351784 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05a830-e3d4-4159-a38c-675c21e47d7b admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.352060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05a830-e3d4-4159-a38c-675c21e47d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.352329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05a830-e3d4-4159-a38c-675c21e47d7b admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.352967 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05a830-e3d4-4159-a38c-675c21e47d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.354995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05a830-e3d4-4159-a38c-675c21e47d7b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.364256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95648380-b09a-4c21-a3ca-0ed712de2335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.365385 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95648380-b09a-4c21-a3ca-0ed712de2335 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.366044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95648380-b09a-4c21-a3ca-0ed712de2335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.366326 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95648380-b09a-4c21-a3ca-0ed712de2335 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.367314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95648380-b09a-4c21-a3ca-0ed712de2335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.369250 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95648380-b09a-4c21-a3ca-0ed712de2335 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.381475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 805/1579] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:37:53 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:53.381475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05a830-e3d4-4159-a38c-675c21e47d7b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.390375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05a830-e3d4-4159-a38c-675c21e47d7b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.395546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fa748-4e1c-4711-bd05-b7dad0482b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.396161 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fa748-4e1c-4711-bd05-b7dad0482b28 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.396445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fa748-4e1c-4711-bd05-b7dad0482b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.396715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fa748-4e1c-4711-bd05-b7dad0482b28 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.397379 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fa748-4e1c-4711-bd05-b7dad0482b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.399301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fa748-4e1c-4711-bd05-b7dad0482b28 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.400197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 775/1580] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:37:53 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:53.402279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95648380-b09a-4c21-a3ca-0ed712de2335 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.423430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 806/1581] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:37:53 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:53.424548 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fa748-4e1c-4711-bd05-b7dad0482b28 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.435130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fa748-4e1c-4711-bd05-b7dad0482b28 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.446620 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:53.478374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95648380-b09a-4c21-a3ca-0ed712de2335 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.485615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56902fef-6595-4333-8d39-710a593eb1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.486245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56902fef-6595-4333-8d39-710a593eb1da admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.486523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56902fef-6595-4333-8d39-710a593eb1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.486794 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56902fef-6595-4333-8d39-710a593eb1da admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.487444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56902fef-6595-4333-8d39-710a593eb1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.489115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56902fef-6595-4333-8d39-710a593eb1da admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.503338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 776/1582] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:53.504820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541a0cb-1e5e-4881-938c-2ee18e587532 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.505127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541a0cb-1e5e-4881-938c-2ee18e587532 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.505405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541a0cb-1e5e-4881-938c-2ee18e587532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.514217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf475b8e-2b04-4959-9258-26c1cddf8e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.515217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf475b8e-2b04-4959-9258-26c1cddf8e81 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.515538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf475b8e-2b04-4959-9258-26c1cddf8e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.516071 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf475b8e-2b04-4959-9258-26c1cddf8e81 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.516773 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf475b8e-2b04-4959-9258-26c1cddf8e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.519014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf475b8e-2b04-4959-9258-26c1cddf8e81 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.550617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-3541a0cb-1e5e-4881-938c-2ee18e587532 None None] MFA Rules not processed for user `1b3bcd7fc8114058884eb8ab09aaa154`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:53.556057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 807/1583] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:37:53 2022] POST /identity/v3/users => generated 296 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:53.556057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf475b8e-2b04-4959-9258-26c1cddf8e81 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.556486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3541a0cb-1e5e-4881-938c-2ee18e587532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:53.564861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf475b8e-2b04-4959-9258-26c1cddf8e81 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.571196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02394d1-a5cc-4e8d-8a20-a8d27bd3b335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.572811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02394d1-a5cc-4e8d-8a20-a8d27bd3b335 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.573317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02394d1-a5cc-4e8d-8a20-a8d27bd3b335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.573774 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02394d1-a5cc-4e8d-8a20-a8d27bd3b335 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.574869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02394d1-a5cc-4e8d-8a20-a8d27bd3b335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.578776 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02394d1-a5cc-4e8d-8a20-a8d27bd3b335 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.611517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 808/1584] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:37:53 2022] POST /identity/v3/users => generated 289 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:53.612431 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02394d1-a5cc-4e8d-8a20-a8d27bd3b335 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.624687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02394d1-a5cc-4e8d-8a20-a8d27bd3b335 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.629690 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1f933-f5d2-4452-9de0-eed6ce96a20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.630459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1f933-f5d2-4452-9de0-eed6ce96a20b admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.630739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1f933-f5d2-4452-9de0-eed6ce96a20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.631032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1f933-f5d2-4452-9de0-eed6ce96a20b admin admin] PATH_INFO: `/v3/projects/514923fc2f8d4253974ff57a05e54224/users/e66c5195039d4528a64d6ab66e753a7f/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.647817 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1f933-f5d2-4452-9de0-eed6ce96a20b admin admin] RBAC: Authorizing `identity:create_grant(project_id=514923fc2f8d4253974ff57a05e54224, user_id=e66c5195039d4528a64d6ab66e753a7f, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.650602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1f933-f5d2-4452-9de0-eed6ce96a20b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.660926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 777/1585] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:53.662489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541a0cb-1e5e-4881-938c-2ee18e587532 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.668020 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 809/1586] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:37:53 2022] PUT /identity/v3/projects/514923fc2f8d4253974ff57a05e54224/users/e66c5195039d4528a64d6ab66e753a7f/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:53.679943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1f933-f5d2-4452-9de0-eed6ce96a20b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.681747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541a0cb-1e5e-4881-938c-2ee18e587532 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.734720 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:53.749994 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227144f-83c9-4c45-9e55-a9efc0563ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.750624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227144f-83c9-4c45-9e55-a9efc0563ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.750904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227144f-83c9-4c45-9e55-a9efc0563ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.751180 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227144f-83c9-4c45-9e55-a9efc0563ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.751830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227144f-83c9-4c45-9e55-a9efc0563ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.753503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227144f-83c9-4c45-9e55-a9efc0563ed8 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.757895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1f933-f5d2-4452-9de0-eed6ce96a20b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.762797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c1d62-19e6-469b-96ba-f344e5a158a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.763602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c1d62-19e6-469b-96ba-f344e5a158a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.763907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c1d62-19e6-469b-96ba-f344e5a158a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.764677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c1d62-19e6-469b-96ba-f344e5a158a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.765372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c1d62-19e6-469b-96ba-f344e5a158a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.767284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c1d62-19e6-469b-96ba-f344e5a158a6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.774926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 778/1587] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:53.775638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227144f-83c9-4c45-9e55-a9efc0563ed8 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.788382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227144f-83c9-4c45-9e55-a9efc0563ed8 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.793378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7503be3-8873-4ec2-8263-56b9b37f9cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.793998 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7503be3-8873-4ec2-8263-56b9b37f9cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.794273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7503be3-8873-4ec2-8263-56b9b37f9cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.794549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7503be3-8873-4ec2-8263-56b9b37f9cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.798522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 810/1588] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:37:53 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:53.798734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7503be3-8873-4ec2-8263-56b9b37f9cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.798834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c1d62-19e6-469b-96ba-f344e5a158a6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.798994 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7503be3-8873-4ec2-8263-56b9b37f9cd9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.806846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c1d62-19e6-469b-96ba-f344e5a158a6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.807686 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 779/1589] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:53.808617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7503be3-8873-4ec2-8263-56b9b37f9cd9 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.814733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd1ed1-4121-41b4-8802-dc4ac28a0285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.815352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd1ed1-4121-41b4-8802-dc4ac28a0285 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.815652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd1ed1-4121-41b4-8802-dc4ac28a0285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.815933 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd1ed1-4121-41b4-8802-dc4ac28a0285 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.816581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd1ed1-4121-41b4-8802-dc4ac28a0285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.818211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd1ed1-4121-41b4-8802-dc4ac28a0285 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.819361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7503be3-8873-4ec2-8263-56b9b37f9cd9 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.824386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1e015-7b03-4952-a403-2397cf5aca8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.825006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1e015-7b03-4952-a403-2397cf5aca8c admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.825314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1e015-7b03-4952-a403-2397cf5aca8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.825589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1e015-7b03-4952-a403-2397cf5aca8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.826235 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1e015-7b03-4952-a403-2397cf5aca8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.828380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1e015-7b03-4952-a403-2397cf5aca8c admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.829478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 811/1590] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:53.830333 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd1ed1-4121-41b4-8802-dc4ac28a0285 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.840294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd1ed1-4121-41b4-8802-dc4ac28a0285 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.844281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 780/1591] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:53.845358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1e015-7b03-4952-a403-2397cf5aca8c admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.848953 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331879d-4b6f-456a-9e7d-131d84c3c8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.849618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331879d-4b6f-456a-9e7d-131d84c3c8dc admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.849895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331879d-4b6f-456a-9e7d-131d84c3c8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.850163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331879d-4b6f-456a-9e7d-131d84c3c8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.850808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331879d-4b6f-456a-9e7d-131d84c3c8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.852393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331879d-4b6f-456a-9e7d-131d84c3c8dc admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.857998 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1e015-7b03-4952-a403-2397cf5aca8c admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.863545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c80ec3-441d-46cc-9c0c-f2980b6f55ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.864226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 812/1592] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:53.864540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c80ec3-441d-46cc-9c0c-f2980b6f55ce admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.865215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c80ec3-441d-46cc-9c0c-f2980b6f55ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.865664 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c80ec3-441d-46cc-9c0c-f2980b6f55ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.866627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98f90c-23cb-4810-a7fa-9f3e1a79a255 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.866732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c80ec3-441d-46cc-9c0c-f2980b6f55ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.867146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98f90c-23cb-4810-a7fa-9f3e1a79a255 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.867595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98f90c-23cb-4810-a7fa-9f3e1a79a255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.870308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c80ec3-441d-46cc-9c0c-f2980b6f55ce admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.902726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-aa98f90c-23cb-4810-a7fa-9f3e1a79a255 None None] MFA Rules not processed for user `e66c5195039d4528a64d6ab66e753a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:53.907729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 781/1593] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:37:53 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:53.907952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-aa98f90c-23cb-4810-a7fa-9f3e1a79a255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:53.908079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c80ec3-441d-46cc-9c0c-f2980b6f55ce admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.922779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:53.972838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c80ec3-441d-46cc-9c0c-f2980b6f55ce admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.977455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 813/1594] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:53.977646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12868aae-024a-43fe-8232-e8fe06e3686e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.978239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12868aae-024a-43fe-8232-e8fe06e3686e admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.978360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98f90c-23cb-4810-a7fa-9f3e1a79a255 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.978536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12868aae-024a-43fe-8232-e8fe06e3686e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.978791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12868aae-024a-43fe-8232-e8fe06e3686e admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:53.979435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12868aae-024a-43fe-8232-e8fe06e3686e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:53.981467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12868aae-024a-43fe-8232-e8fe06e3686e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:53.990150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98f90c-23cb-4810-a7fa-9f3e1a79a255 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:53.993116 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 782/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:53.993116 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12868aae-024a-43fe-8232-e8fe06e3686e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:53.998146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea4853-4a11-4f82-9ab2-9b6ef6ecf249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:53.998798 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea4853-4a11-4f82-9ab2-9b6ef6ecf249 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:53.999076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea4853-4a11-4f82-9ab2-9b6ef6ecf249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:53.999355 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea4853-4a11-4f82-9ab2-9b6ef6ecf249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.000007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea4853-4a11-4f82-9ab2-9b6ef6ecf249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.001628 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea4853-4a11-4f82-9ab2-9b6ef6ecf249 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.003144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12868aae-024a-43fe-8232-e8fe06e3686e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.007767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb90b2-1637-403e-82a0-c38dd5fe2897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.008364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb90b2-1637-403e-82a0-c38dd5fe2897 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.008637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb90b2-1637-403e-82a0-c38dd5fe2897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.008903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb90b2-1637-403e-82a0-c38dd5fe2897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.009571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb90b2-1637-403e-82a0-c38dd5fe2897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.011153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb90b2-1637-403e-82a0-c38dd5fe2897 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.012168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 814/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:54.012985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea4853-4a11-4f82-9ab2-9b6ef6ecf249 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.026859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 783/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:54.026859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb90b2-1637-403e-82a0-c38dd5fe2897 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.027145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea4853-4a11-4f82-9ab2-9b6ef6ecf249 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.031231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367302f-1ecb-4eb1-88cb-2dc920ea4b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.031901 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367302f-1ecb-4eb1-88cb-2dc920ea4b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.032182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367302f-1ecb-4eb1-88cb-2dc920ea4b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.032460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367302f-1ecb-4eb1-88cb-2dc920ea4b7e admin admin] PATH_INFO: `/v3/projects/dc7abcbb99b7466fb1581e1010b4a7e1/users/9369796f4c5b41d6b9fe1e786fbd4b38/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.035343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb90b2-1637-403e-82a0-c38dd5fe2897 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.041047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce32baf-f7a7-4012-972b-1b8df0e23dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.041610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce32baf-f7a7-4012-972b-1b8df0e23dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.041884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce32baf-f7a7-4012-972b-1b8df0e23dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.042152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce32baf-f7a7-4012-972b-1b8df0e23dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.042822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce32baf-f7a7-4012-972b-1b8df0e23dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.044421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce32baf-f7a7-4012-972b-1b8df0e23dc3 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.050088 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367302f-1ecb-4eb1-88cb-2dc920ea4b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7abcbb99b7466fb1581e1010b4a7e1, user_id=9369796f4c5b41d6b9fe1e786fbd4b38, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.052846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367302f-1ecb-4eb1-88cb-2dc920ea4b7e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.054924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 784/1598] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:54.058309 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce32baf-f7a7-4012-972b-1b8df0e23dc3 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.066767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce32baf-f7a7-4012-972b-1b8df0e23dc3 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.068005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 815/1599] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:37:54 2022] PUT /identity/v3/projects/dc7abcbb99b7466fb1581e1010b4a7e1/users/9369796f4c5b41d6b9fe1e786fbd4b38/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:54.068806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367302f-1ecb-4eb1-88cb-2dc920ea4b7e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.129952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe7023-3fa5-4f78-8a4a-2ac26c2f5b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.130571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe7023-3fa5-4f78-8a4a-2ac26c2f5b8e admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.130852 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe7023-3fa5-4f78-8a4a-2ac26c2f5b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.131125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe7023-3fa5-4f78-8a4a-2ac26c2f5b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.131772 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe7023-3fa5-4f78-8a4a-2ac26c2f5b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.133429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe7023-3fa5-4f78-8a4a-2ac26c2f5b8e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.138179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367302f-1ecb-4eb1-88cb-2dc920ea4b7e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.143939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c8ade-cffa-46a3-8b80-aa82a084cd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.144547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c8ade-cffa-46a3-8b80-aa82a084cd6f admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.144823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c8ade-cffa-46a3-8b80-aa82a084cd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.145105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c8ade-cffa-46a3-8b80-aa82a084cd6f admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.145234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 785/1600] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:54.145823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c8ade-cffa-46a3-8b80-aa82a084cd6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.145963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe7023-3fa5-4f78-8a4a-2ac26c2f5b8e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.147732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c8ade-cffa-46a3-8b80-aa82a084cd6f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.158239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe7023-3fa5-4f78-8a4a-2ac26c2f5b8e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.164004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c733b6-7672-45cc-a87b-a453c014d3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.164712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c733b6-7672-45cc-a87b-a453c014d3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.164990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c733b6-7672-45cc-a87b-a453c014d3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.165290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c733b6-7672-45cc-a87b-a453c014d3b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.165929 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c733b6-7672-45cc-a87b-a453c014d3b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.168141 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c733b6-7672-45cc-a87b-a453c014d3b9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.172877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 816/1601] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:37:54 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:54.173747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c8ade-cffa-46a3-8b80-aa82a084cd6f admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.181132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:54.189181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c8ade-cffa-46a3-8b80-aa82a084cd6f admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.200505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d75bb-1515-4614-91d1-e9b87a8c2cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.201404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d75bb-1515-4614-91d1-e9b87a8c2cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.202353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d75bb-1515-4614-91d1-e9b87a8c2cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.202733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d75bb-1515-4614-91d1-e9b87a8c2cbc admin admin] PATH_INFO: `/v3/projects/e7e1bfaf89844022baefa9fa51128e9a/users/bbd861e3414c4b5f9f9a68f3f303a9f2/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.208972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 786/1602] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:37:54 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:54.209865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c733b6-7672-45cc-a87b-a453c014d3b9 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.220827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d75bb-1515-4614-91d1-e9b87a8c2cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e1bfaf89844022baefa9fa51128e9a, user_id=bbd861e3414c4b5f9f9a68f3f303a9f2, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.223739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d75bb-1515-4614-91d1-e9b87a8c2cbc admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.238398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 817/1603] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:37:54 2022] PUT /identity/v3/projects/e7e1bfaf89844022baefa9fa51128e9a/users/bbd861e3414c4b5f9f9a68f3f303a9f2/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:54.238949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d75bb-1515-4614-91d1-e9b87a8c2cbc admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.295110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c733b6-7672-45cc-a87b-a453c014d3b9 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.302013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:54.319051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d75bb-1515-4614-91d1-e9b87a8c2cbc admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.324055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3bec3-68b1-4a3d-a393-20681837e0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.324728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3bec3-68b1-4a3d-a393-20681837e0ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.325004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3bec3-68b1-4a3d-a393-20681837e0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.325311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3bec3-68b1-4a3d-a393-20681837e0ed admin admin] PATH_INFO: `/v3/projects/aaf9ee385c3f44049510466e54c3eac5/users/0c39ace862664bf5b0c8130806ae27d9/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.342403 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3bec3-68b1-4a3d-a393-20681837e0ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaf9ee385c3f44049510466e54c3eac5, user_id=0c39ace862664bf5b0c8130806ae27d9, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.345047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3bec3-68b1-4a3d-a393-20681837e0ed admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.357496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c0426-0519-44aa-9410-14f1b8940cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.358177 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c0426-0519-44aa-9410-14f1b8940cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.358454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c0426-0519-44aa-9410-14f1b8940cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.358722 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c0426-0519-44aa-9410-14f1b8940cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.359368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c0426-0519-44aa-9410-14f1b8940cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.359575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 818/1604] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:37:54 2022] PUT /identity/v3/projects/aaf9ee385c3f44049510466e54c3eac5/users/0c39ace862664bf5b0c8130806ae27d9/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:54.360242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3bec3-68b1-4a3d-a393-20681837e0ed admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.361026 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c0426-0519-44aa-9410-14f1b8940cd0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.373165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 787/1605] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:54.375186 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563eea6-e9f6-4528-a359-1a5e8295c381 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.375469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563eea6-e9f6-4528-a359-1a5e8295c381 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.375755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563eea6-e9f6-4528-a359-1a5e8295c381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.397529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-0563eea6-e9f6-4528-a359-1a5e8295c381 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:54.398775 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-0563eea6-e9f6-4528-a359-1a5e8295c381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:54.443895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3bec3-68b1-4a3d-a393-20681837e0ed admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.448890 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbff56-aab6-42a6-ad2c-311197234be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.449601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbff56-aab6-42a6-ad2c-311197234be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.449888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbff56-aab6-42a6-ad2c-311197234be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.450215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbff56-aab6-42a6-ad2c-311197234be2 admin admin] PATH_INFO: `/v3/projects/42699952850d49fdafb670aeb5f613ed/users/7d712e201f2c47a8a48b6e24559e0d60/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.466752 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbff56-aab6-42a6-ad2c-311197234be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42699952850d49fdafb670aeb5f613ed, user_id=7d712e201f2c47a8a48b6e24559e0d60, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.467773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 788/1606] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:54.469513 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbff56-aab6-42a6-ad2c-311197234be2 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.471913 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563eea6-e9f6-4528-a359-1a5e8295c381 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.482758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563eea6-e9f6-4528-a359-1a5e8295c381 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.488104 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 819/1607] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:37:54 2022] PUT /identity/v3/projects/42699952850d49fdafb670aeb5f613ed/users/7d712e201f2c47a8a48b6e24559e0d60/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:54.488104 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbff56-aab6-42a6-ad2c-311197234be2 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.517129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:54.556444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993f663-1454-4502-a95e-9dae68d62ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.557030 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f663-1454-4502-a95e-9dae68d62ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.557425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f663-1454-4502-a95e-9dae68d62ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.557703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993f663-1454-4502-a95e-9dae68d62ff9 admin admin] PATH_INFO: `/v3/projects/d3e65d28e5664025978902943452f338/users/fe0fcdeafae645a582afcee3dde79ebc/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.559141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbff56-aab6-42a6-ad2c-311197234be2 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.564407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8661b-9f03-4d5c-a049-0d66427fe9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.565027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8661b-9f03-4d5c-a049-0d66427fe9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.565334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8661b-9f03-4d5c-a049-0d66427fe9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.565611 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8661b-9f03-4d5c-a049-0d66427fe9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.566257 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8661b-9f03-4d5c-a049-0d66427fe9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.567846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8661b-9f03-4d5c-a049-0d66427fe9b6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.574581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993f663-1454-4502-a95e-9dae68d62ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e65d28e5664025978902943452f338, user_id=fe0fcdeafae645a582afcee3dde79ebc, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.575618 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: WARNING py.warnings [None req-c993f663-1454-4502-a95e-9dae68d62ff9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:37:54.575618 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: warnings.warn(msg) Sep 28 08:37:54.575618 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: Sep 28 08:37:54.577659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993f663-1454-4502-a95e-9dae68d62ff9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.578697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 820/1608] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:54.579528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8661b-9f03-4d5c-a049-0d66427fe9b6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.591989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8661b-9f03-4d5c-a049-0d66427fe9b6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.597113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 789/1609] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:37:54 2022] PUT /identity/v3/projects/d3e65d28e5664025978902943452f338/users/fe0fcdeafae645a582afcee3dde79ebc/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:54.598480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a3166-9e62-431f-9add-e2602654ee2e None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.598773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a3166-9e62-431f-9add-e2602654ee2e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.599050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a3166-9e62-431f-9add-e2602654ee2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.627900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-790a3166-9e62-431f-9add-e2602654ee2e None None] MFA Rules not processed for user `bbd861e3414c4b5f9f9a68f3f303a9f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:54.629151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-790a3166-9e62-431f-9add-e2602654ee2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:54.646267 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:54.657874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8266140-6b51-4c5f-91af-0b46407eaf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.658553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8266140-6b51-4c5f-91af-0b46407eaf53 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.658864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8266140-6b51-4c5f-91af-0b46407eaf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.659144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8266140-6b51-4c5f-91af-0b46407eaf53 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.659784 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8266140-6b51-4c5f-91af-0b46407eaf53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.661750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8266140-6b51-4c5f-91af-0b46407eaf53 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.692457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 821/1610] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:37:54 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:37:54.692457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dda24-a63e-4e2c-bb28-cbce4c41e596 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.692457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dda24-a63e-4e2c-bb28-cbce4c41e596 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.692852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635dda24-a63e-4e2c-bb28-cbce4c41e596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.705208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 790/1611] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:54.706434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a3166-9e62-431f-9add-e2602654ee2e None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.721193 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a3166-9e62-431f-9add-e2602654ee2e None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.726273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae7946-991e-4538-86c4-287fae6dca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.726944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae7946-991e-4538-86c4-287fae6dca27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.727224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae7946-991e-4538-86c4-287fae6dca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.727495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae7946-991e-4538-86c4-287fae6dca27 admin admin] PATH_INFO: `/v3/projects/6bc96052a69e409498462a1e4d499910/users/7e3b9ceb95374f1dbf356122c8ed463c/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.736669 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-635dda24-a63e-4e2c-bb28-cbce4c41e596 None None] MFA Rules not processed for user `0c39ace862664bf5b0c8130806ae27d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:54.738325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-635dda24-a63e-4e2c-bb28-cbce4c41e596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:54.745073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae7946-991e-4538-86c4-287fae6dca27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bc96052a69e409498462a1e4d499910, user_id=7e3b9ceb95374f1dbf356122c8ed463c, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.748044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae7946-991e-4538-86c4-287fae6dca27 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.770932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 791/1612] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:37:54 2022] PUT /identity/v3/projects/6bc96052a69e409498462a1e4d499910/users/7e3b9ceb95374f1dbf356122c8ed463c/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:54.772214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae7946-991e-4538-86c4-287fae6dca27 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.818267 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 822/1613] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:54.819464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20355787-ecd5-45da-ae06-eb0a716716ae None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.819791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20355787-ecd5-45da-ae06-eb0a716716ae None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.820081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20355787-ecd5-45da-ae06-eb0a716716ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.835907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae7946-991e-4538-86c4-287fae6dca27 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.839847 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832e57d-4b93-4a76-b4be-7e667f851fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.840948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-20355787-ecd5-45da-ae06-eb0a716716ae None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:54.841754 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832e57d-4b93-4a76-b4be-7e667f851fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.842495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832e57d-4b93-4a76-b4be-7e667f851fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.842619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-20355787-ecd5-45da-ae06-eb0a716716ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:54.842837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832e57d-4b93-4a76-b4be-7e667f851fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.880568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:54.909652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 823/1614] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:54.910673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20355787-ecd5-45da-ae06-eb0a716716ae None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.914984 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832e57d-4b93-4a76-b4be-7e667f851fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.916450 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832e57d-4b93-4a76-b4be-7e667f851fb1 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.921942 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20355787-ecd5-45da-ae06-eb0a716716ae None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.928868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 792/1615] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:37:54.929178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4982e2-2fbe-4520-8d50-f6e04fe9cbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:54.929178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4982e2-2fbe-4520-8d50-f6e04fe9cbf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.929488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4982e2-2fbe-4520-8d50-f6e04fe9cbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.931820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace57553-bf03-493a-aa0b-b57ea5f4acc4 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.931971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4982e2-2fbe-4520-8d50-f6e04fe9cbf8 admin admin] PATH_INFO: `/v3/projects/850b340b28b049429059f6d4ddf2833d/users/a6b3a36268114ef0bdcefaf2740d004f/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.932129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace57553-bf03-493a-aa0b-b57ea5f4acc4 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.932129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace57553-bf03-493a-aa0b-b57ea5f4acc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.951227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ace57553-bf03-493a-aa0b-b57ea5f4acc4 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:54.951586 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4982e2-2fbe-4520-8d50-f6e04fe9cbf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=850b340b28b049429059f6d4ddf2833d, user_id=a6b3a36268114ef0bdcefaf2740d004f, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:54.952488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ace57553-bf03-493a-aa0b-b57ea5f4acc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:54.954891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4982e2-2fbe-4520-8d50-f6e04fe9cbf8 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:54.961377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 793/1616] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:54.962305 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace57553-bf03-493a-aa0b-b57ea5f4acc4 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:54.976009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 824/1617] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:37:54 2022] PUT /identity/v3/projects/850b340b28b049429059f6d4ddf2833d/users/a6b3a36268114ef0bdcefaf2740d004f/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:54.978683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8dd4f-dc20-40e7-97ee-113e24cee445 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.978981 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8dd4f-dc20-40e7-97ee-113e24cee445 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.979283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8dd4f-dc20-40e7-97ee-113e24cee445 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:54.980717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 825/1618] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 08:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:37:54.980966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace57553-bf03-493a-aa0b-b57ea5f4acc4 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:54.981861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ccd44-ae41-449e-96ab-0e83415b1d48 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:54.982219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ccd44-ae41-449e-96ab-0e83415b1d48 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:54.982645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ccd44-ae41-449e-96ab-0e83415b1d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.007600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-fa2ccd44-ae41-449e-96ab-0e83415b1d48 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:55.008840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-fa2ccd44-ae41-449e-96ab-0e83415b1d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:55.056930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad077e6d-68b6-466c-bb3d-141a093fa9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:55.059455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad077e6d-68b6-466c-bb3d-141a093fa9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.060240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:55.060681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad077e6d-68b6-466c-bb3d-141a093fa9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.060960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad077e6d-68b6-466c-bb3d-141a093fa9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.062693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad077e6d-68b6-466c-bb3d-141a093fa9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:55.065923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad077e6d-68b6-466c-bb3d-141a093fa9d7 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:55.085392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 826/1619] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:55.086764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4177813-4784-4c26-a391-eb947395231e None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.087051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4177813-4784-4c26-a391-eb947395231e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.087323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4177813-4784-4c26-a391-eb947395231e None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.088562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 794/1620] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:55.090816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad077e6d-68b6-466c-bb3d-141a093fa9d7 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:55.093860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 827/1621] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 08:37:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:37:55.094721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4177813-4784-4c26-a391-eb947395231e None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:55.096801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d4177813-4784-4c26-a391-eb947395231e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:55.163755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad077e6d-68b6-466c-bb3d-141a093fa9d7 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:55.168667 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6a3b1-e717-4e36-ac05-0b824c04ba1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:55.169334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6a3b1-e717-4e36-ac05-0b824c04ba1d service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.169774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6a3b1-e717-4e36-ac05-0b824c04ba1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.169942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6a3b1-e717-4e36-ac05-0b824c04ba1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.182976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6a3b1-e717-4e36-ac05-0b824c04ba1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:55.185776 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6a3b1-e717-4e36-ac05-0b824c04ba1d service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:55.197734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 795/1622] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:37:55.198424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6a3b1-e717-4e36-ac05-0b824c04ba1d service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:55.210011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6a3b1-e717-4e36-ac05-0b824c04ba1d service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:55.212481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4177813-4784-4c26-a391-eb947395231e None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:55.216657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f20c6-2530-4ede-8d76-a6cb1b5a1055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:55.217168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc85ed8-b170-4638-8402-69046241a1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:55.217355 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f20c6-2530-4ede-8d76-a6cb1b5a1055 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.218302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f20c6-2530-4ede-8d76-a6cb1b5a1055 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.218471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc85ed8-b170-4638-8402-69046241a1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.218642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f20c6-2530-4ede-8d76-a6cb1b5a1055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.218807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc85ed8-b170-4638-8402-69046241a1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.218968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc85ed8-b170-4638-8402-69046241a1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.219765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc85ed8-b170-4638-8402-69046241a1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:55.221590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc85ed8-b170-4638-8402-69046241a1d8 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:55.228860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f20c6-2530-4ede-8d76-a6cb1b5a1055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:55.230403 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f20c6-2530-4ede-8d76-a6cb1b5a1055 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:55.236913 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 796/1623] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:55.237982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8528d52-4211-4a0d-8acc-349c3276751f None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.238357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8528d52-4211-4a0d-8acc-349c3276751f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.238724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8528d52-4211-4a0d-8acc-349c3276751f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.245200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 828/1624] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:37:55.246163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f20c6-2530-4ede-8d76-a6cb1b5a1055 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:55.257517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f20c6-2530-4ede-8d76-a6cb1b5a1055 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:55.263203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa4739-1174-4bb2-b194-72d4922bf9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:55.263891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4739-1174-4bb2-b194-72d4922bf9b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.264726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4739-1174-4bb2-b194-72d4922bf9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.265002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa4739-1174-4bb2-b194-72d4922bf9b7 admin admin] PATH_INFO: `/v3/projects/c79a8d7386f74e0eab3332dceaff5876/users/0e673fd5119842be8528edb65e4d339d/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.297807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa4739-1174-4bb2-b194-72d4922bf9b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79a8d7386f74e0eab3332dceaff5876, user_id=0e673fd5119842be8528edb65e4d339d, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:55.302074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa4739-1174-4bb2-b194-72d4922bf9b7 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:55.306957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-a8528d52-4211-4a0d-8acc-349c3276751f None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:55.308600 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a8528d52-4211-4a0d-8acc-349c3276751f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:55.330939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 829/1625] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:37:55 2022] PUT /identity/v3/projects/c79a8d7386f74e0eab3332dceaff5876/users/0e673fd5119842be8528edb65e4d339d/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:55.332428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa4739-1174-4bb2-b194-72d4922bf9b7 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:55.376684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:55.460489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 797/1626] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 08:37:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:55.463663 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29472050-ec88-464d-b3e0-545790f10667 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.464597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29472050-ec88-464d-b3e0-545790f10667 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.465530 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29472050-ec88-464d-b3e0-545790f10667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.481083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa4739-1174-4bb2-b194-72d4922bf9b7 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:55.493485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba6a1d-d382-48cc-a863-f4d5d887dcc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:55.494479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6a1d-d382-48cc-a863-f4d5d887dcc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.494949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6a1d-d382-48cc-a863-f4d5d887dcc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.497024 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6a1d-d382-48cc-a863-f4d5d887dcc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.504179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-29472050-ec88-464d-b3e0-545790f10667 None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:55.505300 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-29472050-ec88-464d-b3e0-545790f10667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:55.609876 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba6a1d-d382-48cc-a863-f4d5d887dcc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:55.612591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba6a1d-d382-48cc-a863-f4d5d887dcc8 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:55.633604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 830/1627] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:37:55.634813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba6a1d-d382-48cc-a863-f4d5d887dcc8 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:55.639617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 798/1628] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 08:37:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:55.649487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29472050-ec88-464d-b3e0-545790f10667 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:55.664662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29472050-ec88-464d-b3e0-545790f10667 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:55.668113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba6a1d-d382-48cc-a863-f4d5d887dcc8 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:55.675705 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960cc68-19f8-4e82-9e77-a259f21e532e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:55.676660 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960cc68-19f8-4e82-9e77-a259f21e532e admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.677031 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960cc68-19f8-4e82-9e77-a259f21e532e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.677652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960cc68-19f8-4e82-9e77-a259f21e532e admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.678418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960cc68-19f8-4e82-9e77-a259f21e532e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:55.678654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a90da2-0393-4fe5-9636-f260d4740357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:55.679657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a90da2-0393-4fe5-9636-f260d4740357 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.679939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a90da2-0393-4fe5-9636-f260d4740357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.680266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960cc68-19f8-4e82-9e77-a259f21e532e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:55.680508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a90da2-0393-4fe5-9636-f260d4740357 admin admin] PATH_INFO: `/v3/projects/850b340b28b049429059f6d4ddf2833d/users/a6b3a36268114ef0bdcefaf2740d004f/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.685291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a90da2-0393-4fe5-9636-f260d4740357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=850b340b28b049429059f6d4ddf2833d, user_id=a6b3a36268114ef0bdcefaf2740d004f, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:55.691500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a90da2-0393-4fe5-9636-f260d4740357 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:55.694113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 799/1629] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:37:55.695074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960cc68-19f8-4e82-9e77-a259f21e532e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:55.715433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 831/1630] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:37:55 2022] PUT /identity/v3/projects/850b340b28b049429059f6d4ddf2833d/users/a6b3a36268114ef0bdcefaf2740d004f/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:55.721962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a90da2-0393-4fe5-9636-f260d4740357 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:55.730003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960cc68-19f8-4e82-9e77-a259f21e532e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:55.731842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:55.863438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0d529-a473-4090-8453-356c791ba0ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:55.864376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0d529-a473-4090-8453-356c791ba0ed service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.865966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0d529-a473-4090-8453-356c791ba0ed service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.866367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0d529-a473-4090-8453-356c791ba0ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:55.903825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a90da2-0393-4fe5-9636-f260d4740357 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:55.914250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225d1a5-9cc1-471b-8719-c9dcb03676cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:55.914989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225d1a5-9cc1-471b-8719-c9dcb03676cc service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:55.915402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225d1a5-9cc1-471b-8719-c9dcb03676cc service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:55.915749 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225d1a5-9cc1-471b-8719-c9dcb03676cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:56.012504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:56.030504 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225d1a5-9cc1-471b-8719-c9dcb03676cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:56.039416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0d529-a473-4090-8453-356c791ba0ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:56.042862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0d529-a473-4090-8453-356c791ba0ed service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:56.047284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225d1a5-9cc1-471b-8719-c9dcb03676cc service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:56.083514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 800/1631] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:37:56.085110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0d529-a473-4090-8453-356c791ba0ed service nova] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:56.107309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 832/1632] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:37:56.114400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77b315-6ddd-4d7d-9087-ab02cce6883a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:56.114856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77b315-6ddd-4d7d-9087-ab02cce6883a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:56.115764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77b315-6ddd-4d7d-9087-ab02cce6883a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:56.257791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-1c77b315-6ddd-4d7d-9087-ab02cce6883a None None] MFA Rules not processed for user `a6b3a36268114ef0bdcefaf2740d004f`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:56.260252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1c77b315-6ddd-4d7d-9087-ab02cce6883a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:56.291529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0d529-a473-4090-8453-356c791ba0ed service nova] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:56.303080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039800f8-9c5d-4f03-aca6-3bd58668584e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:56.303902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039800f8-9c5d-4f03-aca6-3bd58668584e admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:56.304222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039800f8-9c5d-4f03-aca6-3bd58668584e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:56.304528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039800f8-9c5d-4f03-aca6-3bd58668584e admin admin] PATH_INFO: `/v3/projects/c79a8d7386f74e0eab3332dceaff5876/users/0e673fd5119842be8528edb65e4d339d/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:56.309694 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039800f8-9c5d-4f03-aca6-3bd58668584e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79a8d7386f74e0eab3332dceaff5876, user_id=0e673fd5119842be8528edb65e4d339d, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:56.316890 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039800f8-9c5d-4f03-aca6-3bd58668584e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:56.357552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 801/1633] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:37:56 2022] PUT /identity/v3/projects/c79a8d7386f74e0eab3332dceaff5876/users/0e673fd5119842be8528edb65e4d339d/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:56.359799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28966b-799c-496e-9ad5-d4cc961fc91f None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:56.359799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28966b-799c-496e-9ad5-d4cc961fc91f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:56.360086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28966b-799c-496e-9ad5-d4cc961fc91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:56.420778 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 833/1634] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:56.431610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-1e28966b-799c-496e-9ad5-d4cc961fc91f None None] MFA Rules not processed for user `0e673fd5119842be8528edb65e4d339d`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:56.434931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77b315-6ddd-4d7d-9087-ab02cce6883a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:56.435161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1e28966b-799c-496e-9ad5-d4cc961fc91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:56.489278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:56.554502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77b315-6ddd-4d7d-9087-ab02cce6883a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:56.648412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 802/1635] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:37:56.662106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28966b-799c-496e-9ad5-d4cc961fc91f None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:56.669389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36805f-86b3-4a7e-8cc3-1f4762489d62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:56.669389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36805f-86b3-4a7e-8cc3-1f4762489d62 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:56.669389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36805f-86b3-4a7e-8cc3-1f4762489d62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:56.669389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36805f-86b3-4a7e-8cc3-1f4762489d62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:56.682028 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28966b-799c-496e-9ad5-d4cc961fc91f None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:56.691843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893c0e1-f84f-4b0b-b361-d12ec88c54ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:56.691843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893c0e1-f84f-4b0b-b361-d12ec88c54ec service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:56.691843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893c0e1-f84f-4b0b-b361-d12ec88c54ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:56.691843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893c0e1-f84f-4b0b-b361-d12ec88c54ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:56.715440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893c0e1-f84f-4b0b-b361-d12ec88c54ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:56.717279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893c0e1-f84f-4b0b-b361-d12ec88c54ec service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:56.741987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 803/1636] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:37:56.745472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68fc4d-7046-46fc-92ed-d7a8cc9bf57a None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:56.747467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68fc4d-7046-46fc-92ed-d7a8cc9bf57a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:56.747790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68fc4d-7046-46fc-92ed-d7a8cc9bf57a None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:56.754876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 804/1637] 10.222.0.52 () {54 vars in 912 bytes} [Wed Sep 28 08:37:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:37:56.756966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd727a-7dbb-47b2-934c-188f8a4c3f79 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:56.757280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd727a-7dbb-47b2-934c-188f8a4c3f79 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:56.757569 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd727a-7dbb-47b2-934c-188f8a4c3f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:56.804422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-8ecd727a-7dbb-47b2-934c-188f8a4c3f79 None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:56.806183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8ecd727a-7dbb-47b2-934c-188f8a4c3f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:56.821143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36805f-86b3-4a7e-8cc3-1f4762489d62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:56.822598 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36805f-86b3-4a7e-8cc3-1f4762489d62 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:56.845799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 834/1638] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:37:56.847162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e5c07-bf74-4e6c-a4f7-70e3c0a24cb6 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:56.847445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e5c07-bf74-4e6c-a4f7-70e3c0a24cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:56.847716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e5c07-bf74-4e6c-a4f7-70e3c0a24cb6 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:56.861420 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 835/1639] 10.222.0.52 () {54 vars in 912 bytes} [Wed Sep 28 08:37:56 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:37:56.862295 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e5c07-bf74-4e6c-a4f7-70e3c0a24cb6 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:56.936766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:56.974010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 805/1640] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:37:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:56.976701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25144037-d57d-474b-af29-004fdc0bc427 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:56.977155 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25144037-d57d-474b-af29-004fdc0bc427 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:56.977495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25144037-d57d-474b-af29-004fdc0bc427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:57.005442 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e5c07-bf74-4e6c-a4f7-70e3c0a24cb6 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:57.014544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-25144037-d57d-474b-af29-004fdc0bc427 None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:57.015872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-25144037-d57d-474b-af29-004fdc0bc427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:57.016672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:57.017432 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:57.017722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:57.018093 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] PATH_INFO: `/v3/users/e66c5195039d4528a64d6ab66e753a7f` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:57.019410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e66c5195039d4528a64d6ab66e753a7f)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:57.024108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:57.026857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 806/1641] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:37:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:57.033552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25144037-d57d-474b-af29-004fdc0bc427 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:57.045653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25144037-d57d-474b-af29-004fdc0bc427 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:57.049874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc5a26-6f67-44db-a8ac-f4ac3b43ab36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:57.051219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc5a26-6f67-44db-a8ac-f4ac3b43ab36 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:57.051742 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc5a26-6f67-44db-a8ac-f4ac3b43ab36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:57.052096 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc5a26-6f67-44db-a8ac-f4ac3b43ab36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:57.068360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc5a26-6f67-44db-a8ac-f4ac3b43ab36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:57.070003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc5a26-6f67-44db-a8ac-f4ac3b43ab36 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:57.101677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 807/1642] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:37:57.102981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc5a26-6f67-44db-a8ac-f4ac3b43ab36 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:57.103831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:57.128249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc5a26-6f67-44db-a8ac-f4ac3b43ab36 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:57.132112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e66c5195039d4528a64d6ab66e753a7f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:57.139113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e1103-831f-4e23-85bb-1e0fbf969f44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:57.139901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e1103-831f-4e23-85bb-1e0fbf969f44 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:57.140367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e1103-831f-4e23-85bb-1e0fbf969f44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:57.141852 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e1103-831f-4e23-85bb-1e0fbf969f44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:57.161517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e66c5195039d4528a64d6ab66e753a7f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:57.163677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e66c5195039d4528a64d6ab66e753a7f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:57.166741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e1103-831f-4e23-85bb-1e0fbf969f44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:57.168476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e1103-831f-4e23-85bb-1e0fbf969f44 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:57.182395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e66c5195039d4528a64d6ab66e753a7f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:57.184592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e66c5195039d4528a64d6ab66e753a7f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:57.210837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 808/1643] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:37:57.220388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 836/1644] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:37:56 2022] DELETE /identity/v3/users/e66c5195039d4528a64d6ab66e753a7f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:57.226233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123233b5-45ad-4c88-82d8-51db221130ea None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:57.226233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123233b5-45ad-4c88-82d8-51db221130ea None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:57.226804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123233b5-45ad-4c88-82d8-51db221130ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:57.271986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-123233b5-45ad-4c88-82d8-51db221130ea None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:57.275591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-123233b5-45ad-4c88-82d8-51db221130ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:57.469054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 809/1645] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:57.484255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:57.486019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:57.639309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7b370-e38e-49b9-a779-7e3e3a3c8fa9 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:57.649698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3920e79-ad8f-49b8-8342-233babdbc562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:57.649698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3920e79-ad8f-49b8-8342-233babdbc562 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:57.649698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3920e79-ad8f-49b8-8342-233babdbc562 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:57.649698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3920e79-ad8f-49b8-8342-233babdbc562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:57.671712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3920e79-ad8f-49b8-8342-233babdbc562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:57.672438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3920e79-ad8f-49b8-8342-233babdbc562 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:57.700750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 837/1646] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:37:57.824299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123233b5-45ad-4c88-82d8-51db221130ea None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:57.826017 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-123233b5-45ad-4c88-82d8-51db221130ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:57.846014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123233b5-45ad-4c88-82d8-51db221130ea None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:57.852421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:57.853219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:57.853573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:57.853916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] PATH_INFO: `/v3/users/0c39ace862664bf5b0c8130806ae27d9` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:57.858797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c39ace862664bf5b0c8130806ae27d9)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:57.860129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: WARNING py.warnings [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:37:57.860129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: warnings.warn(msg) Sep 28 08:37:57.860129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: Sep 28 08:37:57.862477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:58.002725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c39ace862664bf5b0c8130806ae27d9'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:58.034772 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c39ace862664bf5b0c8130806ae27d9'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:58.037396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c39ace862664bf5b0c8130806ae27d9'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:58.038731 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c39ace862664bf5b0c8130806ae27d9'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:58.066394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c39ace862664bf5b0c8130806ae27d9'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:58.136742 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 810/1647] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:37:57 2022] DELETE /identity/v3/users/0c39ace862664bf5b0c8130806ae27d9 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:58.141699 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a894-863a-44bc-a515-ea8acffe278e None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:58.142317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a894-863a-44bc-a515-ea8acffe278e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:58.142317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a894-863a-44bc-a515-ea8acffe278e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:58.184799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-ed18a894-863a-44bc-a515-ea8acffe278e None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:58.191708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ed18a894-863a-44bc-a515-ea8acffe278e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:58.251476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:58.253622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:58.315955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:58.335573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 838/1648] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:58.362881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18a894-863a-44bc-a515-ea8acffe278e None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:58.366090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ed18a894-863a-44bc-a515-ea8acffe278e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:58.429117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad921f-d10c-4193-96d7-5483a74d4d59 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:58.440966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:58.442189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:58.442650 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:58.443101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] PATH_INFO: `/v3/users/1b3bcd7fc8114058884eb8ab09aaa154` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:58.448427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3bcd7fc8114058884eb8ab09aaa154)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:58.451763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:58.481423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18a894-863a-44bc-a515-ea8acffe278e None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:58.497130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb4f68-bcf1-4237-910b-6c9a5b3d2124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:58.497130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb4f68-bcf1-4237-910b-6c9a5b3d2124 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:58.497130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb4f68-bcf1-4237-910b-6c9a5b3d2124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:58.497536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb4f68-bcf1-4237-910b-6c9a5b3d2124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:58.549711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:58.568262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb4f68-bcf1-4237-910b-6c9a5b3d2124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:58.572404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb4f68-bcf1-4237-910b-6c9a5b3d2124 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:58.595024 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 839/1649] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:37:58.680589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3bcd7fc8114058884eb8ab09aaa154'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:58.703228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3bcd7fc8114058884eb8ab09aaa154'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:58.705451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3bcd7fc8114058884eb8ab09aaa154'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:58.707176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3bcd7fc8114058884eb8ab09aaa154'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:58.740523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3bcd7fc8114058884eb8ab09aaa154'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:58.820657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 811/1650] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:37:58 2022] DELETE /identity/v3/users/1b3bcd7fc8114058884eb8ab09aaa154 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:37:58.825084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcbc28-7810-40f8-a42d-e01121cbb552 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:58.825651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcbc28-7810-40f8-a42d-e01121cbb552 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:58.826148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dcbc28-7810-40f8-a42d-e01121cbb552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:58.849719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-c7dcbc28-7810-40f8-a42d-e01121cbb552 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:37:58.851212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c7dcbc28-7810-40f8-a42d-e01121cbb552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:58.939939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 840/1651] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:37:59.026793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:59.028315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:59.187518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b885a0-3fba-4689-af12-31b6fdf1cbcc admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:59.203276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcecdf-b640-47b0-b900-8882e6056785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:59.204380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcecdf-b640-47b0-b900-8882e6056785 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:59.205016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcecdf-b640-47b0-b900-8882e6056785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:59.205414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcecdf-b640-47b0-b900-8882e6056785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:59.224864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcecdf-b640-47b0-b900-8882e6056785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:59.228159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcecdf-b640-47b0-b900-8882e6056785 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:59.260077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 812/1652] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:37:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:37:59.374842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dcbc28-7810-40f8-a42d-e01121cbb552 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:59.381976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c7dcbc28-7810-40f8-a42d-e01121cbb552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:37:59.398390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dcbc28-7810-40f8-a42d-e01121cbb552 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:59.409949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:59.410977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:59.411470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:59.411926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] PATH_INFO: `/v3/projects/514923fc2f8d4253974ff57a05e54224` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:59.413847 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=514923fc2f8d4253974ff57a05e54224)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:59.415518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: WARNING py.warnings [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 08:37:59.415518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: warnings.warn(msg) Sep 28 08:37:59.415518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: Sep 28 08:37:59.417649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:37:59.548869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '514923fc2f8d4253974ff57a05e54224', 'request_id': 'req-f540700e-f21f-4b59-9656-4fcbf8bd6368'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:59.562616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '514923fc2f8d4253974ff57a05e54224', 'request_id': 'req-f540700e-f21f-4b59-9656-4fcbf8bd6368'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:59.640563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '514923fc2f8d4253974ff57a05e54224', 'request_id': 'req-f540700e-f21f-4b59-9656-4fcbf8bd6368'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:59.662535 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] The token cache is being invalidate because project 514923fc2f8d4253974ff57a05e54224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:37:59.663024 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:59.664138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:37:59.667399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 841/1653] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:37:59 2022] DELETE /identity/v3/projects/514923fc2f8d4253974ff57a05e54224 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:37:59.671507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcecdf-b640-47b0-b900-8882e6056785 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:37:59.687469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcecdf-b640-47b0-b900-8882e6056785 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:37:59.839026 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:37:59.871187 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:37:59.871905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:37:59.872204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:37:59.872499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] PATH_INFO: `/v3/users/0e673fd5119842be8528edb65e4d339d` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:37:59.874494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e673fd5119842be8528edb65e4d339d)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:37:59.883275 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:00.079266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e673fd5119842be8528edb65e4d339d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:00.094283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e673fd5119842be8528edb65e4d339d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:00.095790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e673fd5119842be8528edb65e4d339d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:00.096647 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e673fd5119842be8528edb65e4d339d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:00.110316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e673fd5119842be8528edb65e4d339d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:00.167554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 813/1654] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:37:59 2022] DELETE /identity/v3/users/0e673fd5119842be8528edb65e4d339d => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:00.569835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:00.571416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:00.597704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540700e-f21f-4b59-9656-4fcbf8bd6368 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:00.649969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:00.652999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:00.758381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:00.759485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:00.759485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:00.759665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d admin admin] PATH_INFO: `/v3/projects/aaf9ee385c3f44049510466e54c3eac5` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:00.761423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaf9ee385c3f44049510466e54c3eac5)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:00.762775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:00.822017 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:00.853035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf7088-ac18-4f16-a17e-a1d6e9bc4b1c admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:00.882602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaf9ee385c3f44049510466e54c3eac5', 'request_id': 'req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:00.903672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaf9ee385c3f44049510466e54c3eac5', 'request_id': 'req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:00.911072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaf9ee385c3f44049510466e54c3eac5', 'request_id': 'req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:00.947007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d admin admin] The token cache is being invalidate because project aaf9ee385c3f44049510466e54c3eac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:00.947339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:00.953816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-e353cf4d-7a0f-405c-abdc-62dd7f5ba54d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:00.960276 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 842/1655] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:00 2022] DELETE /identity/v3/projects/aaf9ee385c3f44049510466e54c3eac5 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:00.961747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:00.961896 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c818f-985b-46a8-93cd-5f5b79cfc5e2 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:00.962195 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c818f-985b-46a8-93cd-5f5b79cfc5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:00.962479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c818f-985b-46a8-93cd-5f5b79cfc5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:00.962692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:00.962973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:00.963240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] PATH_INFO: `/v3/projects/9a008845495a4667aded1f8a6f191b3f` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:00.964726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a008845495a4667aded1f8a6f191b3f)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:00.983143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:01.011111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-054c818f-985b-46a8-93cd-5f5b79cfc5e2 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:01.012430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-054c818f-985b-46a8-93cd-5f5b79cfc5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:01.031753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:01.043334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 843/1656] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:01.054738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff3019-219f-4c9e-83f1-b99c5e330e78 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:01.055587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff3019-219f-4c9e-83f1-b99c5e330e78 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:01.056070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff3019-219f-4c9e-83f1-b99c5e330e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:01.115289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-78ff3019-219f-4c9e-83f1-b99c5e330e78 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:01.116620 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-78ff3019-219f-4c9e-83f1-b99c5e330e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:01.148028 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a008845495a4667aded1f8a6f191b3f', 'request_id': 'req-b2d109c5-0b50-487e-91c1-3f3f94daae2a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:01.168137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a008845495a4667aded1f8a6f191b3f', 'request_id': 'req-b2d109c5-0b50-487e-91c1-3f3f94daae2a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:01.189024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a008845495a4667aded1f8a6f191b3f', 'request_id': 'req-b2d109c5-0b50-487e-91c1-3f3f94daae2a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:01.256645 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] The token cache is being invalidate because project 9a008845495a4667aded1f8a6f191b3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:01.256645 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:01.257246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:01.259652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 814/1657] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:00 2022] DELETE /identity/v3/projects/9a008845495a4667aded1f8a6f191b3f => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:01.266312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:01.270038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:01.290668 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 844/1658] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:01.296697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff3019-219f-4c9e-83f1-b99c5e330e78 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:01.309756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d109c5-0b50-487e-91c1-3f3f94daae2a admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:01.313680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff3019-219f-4c9e-83f1-b99c5e330e78 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:01.326420 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a564f-a2a5-4ab1-8e3a-ffac85924f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:01.327339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a564f-a2a5-4ab1-8e3a-ffac85924f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:01.327740 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a564f-a2a5-4ab1-8e3a-ffac85924f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:01.328037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:01.328174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a564f-a2a5-4ab1-8e3a-ffac85924f67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:01.328948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a564f-a2a5-4ab1-8e3a-ffac85924f67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:38:01.329286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:01.329869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:01.330106 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a564f-a2a5-4ab1-8e3a-ffac85924f67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:01.330480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] PATH_INFO: `/v3/users/a6b3a36268114ef0bdcefaf2740d004f` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:01.332162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6b3a36268114ef0bdcefaf2740d004f)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:01.334042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a564f-a2a5-4ab1-8e3a-ffac85924f67 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:01.334715 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:01.368616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 845/1659] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:38:01.374189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a564f-a2a5-4ab1-8e3a-ffac85924f67 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:01.392923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a564f-a2a5-4ab1-8e3a-ffac85924f67 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:01.412305 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2bbd30-0f8b-414d-a09c-e7300588be74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:01.413170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2bbd30-0f8b-414d-a09c-e7300588be74 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:01.413786 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2bbd30-0f8b-414d-a09c-e7300588be74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:01.414162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2bbd30-0f8b-414d-a09c-e7300588be74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:01.416868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2bbd30-0f8b-414d-a09c-e7300588be74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:01.423533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2bbd30-0f8b-414d-a09c-e7300588be74 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:01.477168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6b3a36268114ef0bdcefaf2740d004f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:01.511813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 846/1660] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:01 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:01.521640 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2bbd30-0f8b-414d-a09c-e7300588be74 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:01.533724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2d2bbd30-0f8b-414d-a09c-e7300588be74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:01.544182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6b3a36268114ef0bdcefaf2740d004f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:01.547020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6b3a36268114ef0bdcefaf2740d004f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:01.548375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6b3a36268114ef0bdcefaf2740d004f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:01.595971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6b3a36268114ef0bdcefaf2740d004f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:01.686028 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 815/1661] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:01 2022] DELETE /identity/v3/users/a6b3a36268114ef0bdcefaf2740d004f => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:01.688595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:01.696844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:01.729976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2bbd30-0f8b-414d-a09c-e7300588be74 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:01.774373 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:01.863831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47654c31-eb65-4297-99d7-5facec2fd864 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:01.884249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e731e9e-751f-4eb5-b7dd-67d075ff0576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:01.884931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e731e9e-751f-4eb5-b7dd-67d075ff0576 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:01.885244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e731e9e-751f-4eb5-b7dd-67d075ff0576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:01.885518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e731e9e-751f-4eb5-b7dd-67d075ff0576 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:01.886210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e731e9e-751f-4eb5-b7dd-67d075ff0576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:01.888426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e731e9e-751f-4eb5-b7dd-67d075ff0576 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:01.892781 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f671d1-231c-427a-991d-2f2f621f756b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:01.893491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f671d1-231c-427a-991d-2f2f621f756b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:01.893778 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f671d1-231c-427a-991d-2f2f621f756b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:01.894052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f671d1-231c-427a-991d-2f2f621f756b admin admin] PATH_INFO: `/v3/projects/c79a8d7386f74e0eab3332dceaff5876` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:01.895290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f671d1-231c-427a-991d-2f2f621f756b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c79a8d7386f74e0eab3332dceaff5876)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:01.901573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f671d1-231c-427a-991d-2f2f621f756b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:01.938788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 816/1662] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:01 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:01.942750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e731e9e-751f-4eb5-b7dd-67d075ff0576 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:01.961704 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e731e9e-751f-4eb5-b7dd-67d075ff0576 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:01.972282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb94277-410a-4bbb-987e-66bc2ee3fbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:01.978727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb94277-410a-4bbb-987e-66bc2ee3fbab admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:01.979438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb94277-410a-4bbb-987e-66bc2ee3fbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:01.984706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb94277-410a-4bbb-987e-66bc2ee3fbab admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:01.986086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb94277-410a-4bbb-987e-66bc2ee3fbab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:01.990388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb94277-410a-4bbb-987e-66bc2ee3fbab admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:01.996007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=78,keystone:DELETE=17,keystone:INSERT=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:02.013747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-60f671d1-231c-427a-991d-2f2f621f756b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c79a8d7386f74e0eab3332dceaff5876', 'request_id': 'req-60f671d1-231c-427a-991d-2f2f621f756b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:02.015406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 817/1663] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:02.020056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb94277-410a-4bbb-987e-66bc2ee3fbab admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:02.030369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-60f671d1-231c-427a-991d-2f2f621f756b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c79a8d7386f74e0eab3332dceaff5876', 'request_id': 'req-60f671d1-231c-427a-991d-2f2f621f756b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:02.050142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb94277-410a-4bbb-987e-66bc2ee3fbab admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:02.053274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-60f671d1-231c-427a-991d-2f2f621f756b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c79a8d7386f74e0eab3332dceaff5876', 'request_id': 'req-60f671d1-231c-427a-991d-2f2f621f756b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:02.081022 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-60f671d1-231c-427a-991d-2f2f621f756b admin admin] The token cache is being invalidate because project c79a8d7386f74e0eab3332dceaff5876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:02.081287 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-60f671d1-231c-427a-991d-2f2f621f756b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:02.082793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-60f671d1-231c-427a-991d-2f2f621f756b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:02.091798 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 847/1664] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:01 2022] DELETE /identity/v3/projects/c79a8d7386f74e0eab3332dceaff5876 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:02.098533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693e0fd-897a-43af-b5fc-3b2d1bac7bd5 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:02.098818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693e0fd-897a-43af-b5fc-3b2d1bac7bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:02.099160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693e0fd-897a-43af-b5fc-3b2d1bac7bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:02.158317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5693e0fd-897a-43af-b5fc-3b2d1bac7bd5 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:02.163898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5693e0fd-897a-43af-b5fc-3b2d1bac7bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:02.280888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e1b44-f361-463c-af15-2495104dda9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:02.281568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e1b44-f361-463c-af15-2495104dda9e admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:02.281854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e1b44-f361-463c-af15-2495104dda9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:02.282162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e1b44-f361-463c-af15-2495104dda9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:02.282805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e1b44-f361-463c-af15-2495104dda9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:02.284763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e1b44-f361-463c-af15-2495104dda9e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:02.321673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 818/1665] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:02.322632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e1b44-f361-463c-af15-2495104dda9e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:02.343430 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a67e1b44-f361-463c-af15-2495104dda9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:02.397288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e1b44-f361-463c-af15-2495104dda9e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:02.413846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be0dce-59d9-4f96-9091-69e5d65a60c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:02.417575 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be0dce-59d9-4f96-9091-69e5d65a60c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:02.417893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 848/1666] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:02.418527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be0dce-59d9-4f96-9091-69e5d65a60c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:02.418971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be0dce-59d9-4f96-9091-69e5d65a60c8 admin admin] PATH_INFO: `/v3/projects/6a472511439940f390b7b40a73e9e1d1/users/45ff264f831e4750b4bd92e90b10ce8d/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:02.427168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ce2fa-23d9-47a3-91b7-8c9a70881283 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:02.427671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ce2fa-23d9-47a3-91b7-8c9a70881283 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:02.428099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ce2fa-23d9-47a3-91b7-8c9a70881283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:02.469730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be0dce-59d9-4f96-9091-69e5d65a60c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a472511439940f390b7b40a73e9e1d1, user_id=45ff264f831e4750b4bd92e90b10ce8d, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:02.476953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be0dce-59d9-4f96-9091-69e5d65a60c8 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:02.485502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-729ce2fa-23d9-47a3-91b7-8c9a70881283 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:02.491790 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-729ce2fa-23d9-47a3-91b7-8c9a70881283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:02.537004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 819/1667] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:02 2022] PUT /identity/v3/projects/6a472511439940f390b7b40a73e9e1d1/users/45ff264f831e4750b4bd92e90b10ce8d/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:02.537452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 849/1668] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:02.565275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd2895-1459-4058-8446-142b9752904a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:02.566411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be0dce-59d9-4f96-9091-69e5d65a60c8 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:02.585912 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd2895-1459-4058-8446-142b9752904a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:02.590025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd2895-1459-4058-8446-142b9752904a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:02.704218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-d0dd2895-1459-4058-8446-142b9752904a None None] MFA Rules not processed for user `45ff264f831e4750b4bd92e90b10ce8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:02.711610 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d0dd2895-1459-4058-8446-142b9752904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:02.852858 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:02.974717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 850/1669] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:02.976086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd2895-1459-4058-8446-142b9752904a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:02.977891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d0dd2895-1459-4058-8446-142b9752904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:03.130751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be0dce-59d9-4f96-9091-69e5d65a60c8 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:03.167798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24421c-8eba-4570-be9a-2cdec258cda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:03.168795 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24421c-8eba-4570-be9a-2cdec258cda7 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:03.169495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24421c-8eba-4570-be9a-2cdec258cda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:03.169938 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24421c-8eba-4570-be9a-2cdec258cda7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:03.171738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24421c-8eba-4570-be9a-2cdec258cda7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:38:03.172268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24421c-8eba-4570-be9a-2cdec258cda7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:03.174977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24421c-8eba-4570-be9a-2cdec258cda7 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:03.182258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd2895-1459-4058-8446-142b9752904a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:03.187864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:03.188541 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:03.188815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:03.189108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] PATH_INFO: `/v3/users/bbd861e3414c4b5f9f9a68f3f303a9f2` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:03.191128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbd861e3414c4b5f9f9a68f3f303a9f2)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:03.192680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:03.236440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 820/1670] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 672 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:38:03.237889 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24421c-8eba-4570-be9a-2cdec258cda7 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:03.254413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24421c-8eba-4570-be9a-2cdec258cda7 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:03.285607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25821900-2242-461d-887b-6c087b53e75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:03.286896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25821900-2242-461d-887b-6c087b53e75b admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:03.287389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25821900-2242-461d-887b-6c087b53e75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:03.287977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25821900-2242-461d-887b-6c087b53e75b admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:03.289583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25821900-2242-461d-887b-6c087b53e75b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:03.292402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25821900-2242-461d-887b-6c087b53e75b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:03.355273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbd861e3414c4b5f9f9a68f3f303a9f2'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:03.378205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 821/1671] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:03 2022] POST /identity/v3/projects => generated 371 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:03.379361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25821900-2242-461d-887b-6c087b53e75b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:03.397251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbd861e3414c4b5f9f9a68f3f303a9f2'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:03.398133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbd861e3414c4b5f9f9a68f3f303a9f2'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:03.423424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbd861e3414c4b5f9f9a68f3f303a9f2'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:03.426938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbd861e3414c4b5f9f9a68f3f303a9f2'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:03.484047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 851/1672] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:02 2022] DELETE /identity/v3/users/bbd861e3414c4b5f9f9a68f3f303a9f2 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:03.486050 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:03.495719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:03.534762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:03.627643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25821900-2242-461d-887b-6c087b53e75b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:03.732277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:03.781409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e3788-a2ec-4da5-8dea-26e76802aade admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:03.791588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20c388-45c9-46d7-b3eb-bd93cffc3525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:03.793515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20c388-45c9-46d7-b3eb-bd93cffc3525 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:03.793954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20c388-45c9-46d7-b3eb-bd93cffc3525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:03.794482 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20c388-45c9-46d7-b3eb-bd93cffc3525 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:03.795848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20c388-45c9-46d7-b3eb-bd93cffc3525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:03.798216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20c388-45c9-46d7-b3eb-bd93cffc3525 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:03.872113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490daf05-0d19-4996-8ef5-063a4cbefb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:03.875598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490daf05-0d19-4996-8ef5-063a4cbefb11 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:03.879017 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490daf05-0d19-4996-8ef5-063a4cbefb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:03.879341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490daf05-0d19-4996-8ef5-063a4cbefb11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:03.882475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490daf05-0d19-4996-8ef5-063a4cbefb11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:03.886865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490daf05-0d19-4996-8ef5-063a4cbefb11 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:03.891466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 852/1673] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:03 2022] POST /identity/v3/users => generated 292 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:03.893136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04920617-1188-4b21-b5f3-3886fe6211fe None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:03.893352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04920617-1188-4b21-b5f3-3886fe6211fe None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:03.894237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04920617-1188-4b21-b5f3-3886fe6211fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:03.936601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-04920617-1188-4b21-b5f3-3886fe6211fe None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:03.938955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-04920617-1188-4b21-b5f3-3886fe6211fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:03.974449 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 822/1674] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:03 2022] POST /identity/v3/projects => generated 385 bytes in 596 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:03.979763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490daf05-0d19-4996-8ef5-063a4cbefb11 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:03.983618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 853/1675] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:03.985437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04920617-1188-4b21-b5f3-3886fe6211fe None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:03.998100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-04920617-1188-4b21-b5f3-3886fe6211fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:04.046093 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04920617-1188-4b21-b5f3-3886fe6211fe None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:04.195418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490daf05-0d19-4996-8ef5-063a4cbefb11 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:04.213371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fe4df-e1dd-4222-9700-070cd34e0b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:04.214024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fe4df-e1dd-4222-9700-070cd34e0b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:04.214308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fe4df-e1dd-4222-9700-070cd34e0b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:04.214583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fe4df-e1dd-4222-9700-070cd34e0b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:04.215245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fe4df-e1dd-4222-9700-070cd34e0b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:04.222298 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fe4df-e1dd-4222-9700-070cd34e0b6d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:04.257712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 823/1676] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:04.258790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fe4df-e1dd-4222-9700-070cd34e0b6d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:04.265835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ff1fe4df-e1dd-4222-9700-070cd34e0b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:04.275344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01ee8-9a81-40a1-909a-7b6c895061e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:04.275601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01ee8-9a81-40a1-909a-7b6c895061e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:04.275697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01ee8-9a81-40a1-909a-7b6c895061e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:04.275932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01ee8-9a81-40a1-909a-7b6c895061e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:04.276600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e01ee8-9a81-40a1-909a-7b6c895061e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:04.278641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e01ee8-9a81-40a1-909a-7b6c895061e4 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:04.344480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 854/1677] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:03 2022] POST /identity/v3/users => generated 293 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:04.345898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01ee8-9a81-40a1-909a-7b6c895061e4 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:04.356600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-94e01ee8-9a81-40a1-909a-7b6c895061e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:04.387430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01ee8-9a81-40a1-909a-7b6c895061e4 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:04.397268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:04.398472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:04.398916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:04.399339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] PATH_INFO: `/v3/projects/850b340b28b049429059f6d4ddf2833d` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:04.404958 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=850b340b28b049429059f6d4ddf2833d)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:04.407247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:04.460687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fe4df-e1dd-4222-9700-070cd34e0b6d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:04.493405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2546b-6091-40dd-8ffd-e67a1fd499d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:04.495243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2546b-6091-40dd-8ffd-e67a1fd499d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:04.495856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2546b-6091-40dd-8ffd-e67a1fd499d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:04.496763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2546b-6091-40dd-8ffd-e67a1fd499d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:04.552869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2546b-6091-40dd-8ffd-e67a1fd499d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:04.556745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2546b-6091-40dd-8ffd-e67a1fd499d8 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:04.594886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '850b340b28b049429059f6d4ddf2833d', 'request_id': 'req-71d65b38-1034-4a70-a4e1-84935132ee67'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:04.613420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 824/1678] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:04.616707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2546b-6091-40dd-8ffd-e67a1fd499d8 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:04.624387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '850b340b28b049429059f6d4ddf2833d', 'request_id': 'req-71d65b38-1034-4a70-a4e1-84935132ee67'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:04.652192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '850b340b28b049429059f6d4ddf2833d', 'request_id': 'req-71d65b38-1034-4a70-a4e1-84935132ee67'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:04.663815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2546b-6091-40dd-8ffd-e67a1fd499d8 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:04.697016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:04.703766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] The token cache is being invalidate because project 850b340b28b049429059f6d4ddf2833d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:04.704087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:04.704820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:04.706626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 855/1679] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:04 2022] DELETE /identity/v3/projects/850b340b28b049429059f6d4ddf2833d => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:04.711700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:04.722684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:04.843218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:04.876947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707d00a-e35b-4065-bd9c-a0d27db68660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:04.879293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707d00a-e35b-4065-bd9c-a0d27db68660 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:04.879498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d65b38-1034-4a70-a4e1-84935132ee67 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:04.879793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707d00a-e35b-4065-bd9c-a0d27db68660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:04.880237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707d00a-e35b-4065-bd9c-a0d27db68660 admin admin] PATH_INFO: `/v3/projects/0e6566acd7774a1c8da0cd1f65560b0e/users/59c1951fab3e4134bb2ab679e2a2b21a/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:04.893197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf48233-a936-48c4-9edb-c9926309ec42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:04.894003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf48233-a936-48c4-9edb-c9926309ec42 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:04.894146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf48233-a936-48c4-9edb-c9926309ec42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:04.894411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf48233-a936-48c4-9edb-c9926309ec42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:04.895017 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf48233-a936-48c4-9edb-c9926309ec42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:04.896655 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf48233-a936-48c4-9edb-c9926309ec42 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:04.927898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707d00a-e35b-4065-bd9c-a0d27db68660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e6566acd7774a1c8da0cd1f65560b0e, user_id=59c1951fab3e4134bb2ab679e2a2b21a, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:04.932087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 856/1680] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:04.932087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098b1dd-0a2d-49a5-a338-318223620626 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:04.932087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098b1dd-0a2d-49a5-a338-318223620626 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:04.932087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098b1dd-0a2d-49a5-a338-318223620626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:04.935310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a707d00a-e35b-4065-bd9c-a0d27db68660 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:04.970539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 825/1681] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:04 2022] PUT /identity/v3/projects/0e6566acd7774a1c8da0cd1f65560b0e/users/59c1951fab3e4134bb2ab679e2a2b21a/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:04.972077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707d00a-e35b-4065-bd9c-a0d27db68660 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:04.981251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-9098b1dd-0a2d-49a5-a338-318223620626 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:04.982618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9098b1dd-0a2d-49a5-a338-318223620626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:05.139096 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 857/1682] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:05.140837 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098b1dd-0a2d-49a5-a338-318223620626 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:05.144209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9098b1dd-0a2d-49a5-a338-318223620626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:05.168517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a707d00a-e35b-4065-bd9c-a0d27db68660 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:05.172992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098b1dd-0a2d-49a5-a338-318223620626 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:05.181638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addf44c-dc66-4179-8bec-d215d197cc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:05.183665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addf44c-dc66-4179-8bec-d215d197cc88 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:05.184512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addf44c-dc66-4179-8bec-d215d197cc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:05.185153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addf44c-dc66-4179-8bec-d215d197cc88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:05.186878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addf44c-dc66-4179-8bec-d215d197cc88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:05.189331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42179727-170b-41fa-8558-8a1fe10de844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:05.189865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addf44c-dc66-4179-8bec-d215d197cc88 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:05.190120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42179727-170b-41fa-8558-8a1fe10de844 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:05.190397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42179727-170b-41fa-8558-8a1fe10de844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:05.190668 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42179727-170b-41fa-8558-8a1fe10de844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:05.191342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42179727-170b-41fa-8558-8a1fe10de844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:05.193076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42179727-170b-41fa-8558-8a1fe10de844 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:05.222248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 826/1683] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:05.229905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166112c-ad47-49cc-8a80-aca5cc0e5dbb None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:05.230619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166112c-ad47-49cc-8a80-aca5cc0e5dbb None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:05.231410 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166112c-ad47-49cc-8a80-aca5cc0e5dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:05.235519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 858/1684] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:05.236915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42179727-170b-41fa-8558-8a1fe10de844 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:05.268486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42179727-170b-41fa-8558-8a1fe10de844 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:05.285352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3b2f1-4fe2-4485-86ce-79bfabf97392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:05.286823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3b2f1-4fe2-4485-86ce-79bfabf97392 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:05.287347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3b2f1-4fe2-4485-86ce-79bfabf97392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:05.287874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3b2f1-4fe2-4485-86ce-79bfabf97392 admin admin] PATH_INFO: `/v3/projects/ded3c4e10415475683af7c59216d5a80/users/f000966ae1fa49db8d936c4d68be9340/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:05.292734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-4166112c-ad47-49cc-8a80-aca5cc0e5dbb None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:05.297567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4166112c-ad47-49cc-8a80-aca5cc0e5dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:05.334459 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 827/1685] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:05.334459 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166112c-ad47-49cc-8a80-aca5cc0e5dbb None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:05.347433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3b2f1-4fe2-4485-86ce-79bfabf97392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded3c4e10415475683af7c59216d5a80, user_id=f000966ae1fa49db8d936c4d68be9340, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:05.352362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3b2f1-4fe2-4485-86ce-79bfabf97392 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:05.363745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166112c-ad47-49cc-8a80-aca5cc0e5dbb None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:05.374135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcef20-5e2b-49bd-bde1-a20f5b91af89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:05.378543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcef20-5e2b-49bd-bde1-a20f5b91af89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:05.381558 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 859/1686] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:05 2022] PUT /identity/v3/projects/ded3c4e10415475683af7c59216d5a80/users/f000966ae1fa49db8d936c4d68be9340/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:05.391105 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcef20-5e2b-49bd-bde1-a20f5b91af89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:05.391472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcef20-5e2b-49bd-bde1-a20f5b91af89 admin admin] PATH_INFO: `/v3/projects/0e6566acd7774a1c8da0cd1f65560b0e/users/59c1951fab3e4134bb2ab679e2a2b21a/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:05.394057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3b2f1-4fe2-4485-86ce-79bfabf97392 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:05.396570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcef20-5e2b-49bd-bde1-a20f5b91af89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e6566acd7774a1c8da0cd1f65560b0e, user_id=59c1951fab3e4134bb2ab679e2a2b21a, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:05.401637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcef20-5e2b-49bd-bde1-a20f5b91af89 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:05.431192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 828/1687] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:05 2022] PUT /identity/v3/projects/0e6566acd7774a1c8da0cd1f65560b0e/users/59c1951fab3e4134bb2ab679e2a2b21a/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:05.434164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de887577-ab23-4475-a7df-b9741924c993 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:05.435179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de887577-ab23-4475-a7df-b9741924c993 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:05.435635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de887577-ab23-4475-a7df-b9741924c993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:05.503665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-de887577-ab23-4475-a7df-b9741924c993 None None] MFA Rules not processed for user `f000966ae1fa49db8d936c4d68be9340`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:05.507830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-de887577-ab23-4475-a7df-b9741924c993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:05.547346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3b2f1-4fe2-4485-86ce-79bfabf97392 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:05.673500 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:05.690011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 829/1688] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:05.693312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d052b31-c5b7-42c8-a9a1-4225d347e829 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:05.693604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d052b31-c5b7-42c8-a9a1-4225d347e829 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:05.694056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d052b31-c5b7-42c8-a9a1-4225d347e829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:05.763069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:05.797372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-3d052b31-c5b7-42c8-a9a1-4225d347e829 None None] MFA Rules not processed for user `59c1951fab3e4134bb2ab679e2a2b21a`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:05.799648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22ec02-4756-4a83-bd1b-efa0168ee9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:05.800596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22ec02-4756-4a83-bd1b-efa0168ee9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:05.801017 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22ec02-4756-4a83-bd1b-efa0168ee9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:05.803330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3d052b31-c5b7-42c8-a9a1-4225d347e829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:05.804482 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22ec02-4756-4a83-bd1b-efa0168ee9d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:05.805645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22ec02-4756-4a83-bd1b-efa0168ee9d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:38:05.806112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22ec02-4756-4a83-bd1b-efa0168ee9d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:05.808921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22ec02-4756-4a83-bd1b-efa0168ee9d1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:05.882200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 860/1689] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:38:05.883168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22ec02-4756-4a83-bd1b-efa0168ee9d1 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:05.918497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22ec02-4756-4a83-bd1b-efa0168ee9d1 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:05.944454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365e9b6-1c26-4814-a714-5015205bd4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:05.945202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365e9b6-1c26-4814-a714-5015205bd4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:05.945416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365e9b6-1c26-4814-a714-5015205bd4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:05.945692 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365e9b6-1c26-4814-a714-5015205bd4b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:05.946345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365e9b6-1c26-4814-a714-5015205bd4b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:05.948303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365e9b6-1c26-4814-a714-5015205bd4b5 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:05.986303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 830/1690] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:05.987206 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d052b31-c5b7-42c8-a9a1-4225d347e829 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:06.007558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d052b31-c5b7-42c8-a9a1-4225d347e829 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:06.023529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26e394-9446-44ff-befe-f3fe948ac9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:06.024222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26e394-9446-44ff-befe-f3fe948ac9a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:06.025336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26e394-9446-44ff-befe-f3fe948ac9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:06.025719 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26e394-9446-44ff-befe-f3fe948ac9a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:06.026754 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26e394-9446-44ff-befe-f3fe948ac9a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:06.028891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26e394-9446-44ff-befe-f3fe948ac9a5 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:06.049974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 861/1691] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:05 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:06.053979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365e9b6-1c26-4814-a714-5015205bd4b5 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:06.058700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4365e9b6-1c26-4814-a714-5015205bd4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:06.086780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 831/1692] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:05 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:06.087498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26e394-9446-44ff-befe-f3fe948ac9a5 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:06.164212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365e9b6-1c26-4814-a714-5015205bd4b5 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:06.199154 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26e394-9446-44ff-befe-f3fe948ac9a5 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:06.211685 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16cfec-ff4e-44ef-b217-7e2d5d04df69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:06.211941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16cfec-ff4e-44ef-b217-7e2d5d04df69 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:06.212065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16cfec-ff4e-44ef-b217-7e2d5d04df69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:06.212673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16cfec-ff4e-44ef-b217-7e2d5d04df69 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:06.213632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16cfec-ff4e-44ef-b217-7e2d5d04df69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:06.216641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16cfec-ff4e-44ef-b217-7e2d5d04df69 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:06.281488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 832/1693] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:06 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:06.285161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16cfec-ff4e-44ef-b217-7e2d5d04df69 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:06.304955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58ad9a-4945-42af-9812-b42ee412b4c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:06.306230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58ad9a-4945-42af-9812-b42ee412b4c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:06.307007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58ad9a-4945-42af-9812-b42ee412b4c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:06.315553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58ad9a-4945-42af-9812-b42ee412b4c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:06.337745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16cfec-ff4e-44ef-b217-7e2d5d04df69 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:06.343393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64463b9d-e755-412c-aa77-48cdfb859c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:06.344025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64463b9d-e755-412c-aa77-48cdfb859c47 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:06.344307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64463b9d-e755-412c-aa77-48cdfb859c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:06.344577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64463b9d-e755-412c-aa77-48cdfb859c47 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:06.345293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64463b9d-e755-412c-aa77-48cdfb859c47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:06.347292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64463b9d-e755-412c-aa77-48cdfb859c47 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:06.377695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 833/1694] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:06 2022] POST /identity/v3/users => generated 282 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:06.377695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64463b9d-e755-412c-aa77-48cdfb859c47 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:06.397505 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64463b9d-e755-412c-aa77-48cdfb859c47 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:06.407877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcf0da-cafa-4f27-b4f5-56398a747a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:06.408532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcf0da-cafa-4f27-b4f5-56398a747a8e admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:06.409063 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcf0da-cafa-4f27-b4f5-56398a747a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:06.409462 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcf0da-cafa-4f27-b4f5-56398a747a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:06.413566 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcf0da-cafa-4f27-b4f5-56398a747a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:06.415862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcf0da-cafa-4f27-b4f5-56398a747a8e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:06.434967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 834/1695] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:06.439963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcf0da-cafa-4f27-b4f5-56398a747a8e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:06.453521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcf0da-cafa-4f27-b4f5-56398a747a8e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:06.459572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39c15e-6282-462c-915b-bea9e615831a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:06.460341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39c15e-6282-462c-915b-bea9e615831a admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:06.460650 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39c15e-6282-462c-915b-bea9e615831a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:06.460927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39c15e-6282-462c-915b-bea9e615831a admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:06.461798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39c15e-6282-462c-915b-bea9e615831a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:06.463496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39c15e-6282-462c-915b-bea9e615831a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:06.479050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 835/1696] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:06.480351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39c15e-6282-462c-915b-bea9e615831a admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:06.512244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39c15e-6282-462c-915b-bea9e615831a admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:06.516849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58ad9a-4945-42af-9812-b42ee412b4c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:06.529212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58ad9a-4945-42af-9812-b42ee412b4c5 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:06.533399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ceb25-43fe-42ae-8282-d5efaca402b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:06.534588 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ceb25-43fe-42ae-8282-d5efaca402b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:06.535067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ceb25-43fe-42ae-8282-d5efaca402b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:06.535550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ceb25-43fe-42ae-8282-d5efaca402b3 admin admin] PATH_INFO: `/v3/projects/d41a39b17be64fc5afcb4ed123c6bdf6/users/484927e9ac594c40bb79637ec43f1fbe/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:06.571009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 862/1697] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:06.571212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:06.572026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58ad9a-4945-42af-9812-b42ee412b4c5 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:06.583833 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ceb25-43fe-42ae-8282-d5efaca402b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d41a39b17be64fc5afcb4ed123c6bdf6, user_id=484927e9ac594c40bb79637ec43f1fbe, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:06.593069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ceb25-43fe-42ae-8282-d5efaca402b3 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:06.599426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58ad9a-4945-42af-9812-b42ee412b4c5 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:06.613932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57c10e-f837-4e55-9fab-495cd35e6256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:06.615479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57c10e-f837-4e55-9fab-495cd35e6256 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:06.615763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57c10e-f837-4e55-9fab-495cd35e6256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:06.616241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57c10e-f837-4e55-9fab-495cd35e6256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:06.618283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57c10e-f837-4e55-9fab-495cd35e6256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:06.620419 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 836/1698] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:06 2022] PUT /identity/v3/projects/d41a39b17be64fc5afcb4ed123c6bdf6/users/484927e9ac594c40bb79637ec43f1fbe/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:06.623453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57c10e-f837-4e55-9fab-495cd35e6256 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:06.625340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ceb25-43fe-42ae-8282-d5efaca402b3 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:06.673599 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 863/1699] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:06.678531 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57c10e-f837-4e55-9fab-495cd35e6256 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:06.697887 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57c10e-f837-4e55-9fab-495cd35e6256 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:06.745576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:06.788050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ceb25-43fe-42ae-8282-d5efaca402b3 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:06.804616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507167a2-bbd8-451d-9e3c-de92fb27f03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:06.806638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507167a2-bbd8-451d-9e3c-de92fb27f03d admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:06.807246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507167a2-bbd8-451d-9e3c-de92fb27f03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:06.809377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507167a2-bbd8-451d-9e3c-de92fb27f03d admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:06.810621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507167a2-bbd8-451d-9e3c-de92fb27f03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:06.814283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507167a2-bbd8-451d-9e3c-de92fb27f03d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:06.844384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 837/1700] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:06.845730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507167a2-bbd8-451d-9e3c-de92fb27f03d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:06.846330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc5bea-4705-4a08-81ac-f310463b91c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:06.847033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc5bea-4705-4a08-81ac-f310463b91c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:06.847331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc5bea-4705-4a08-81ac-f310463b91c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:06.847681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc5bea-4705-4a08-81ac-f310463b91c0 admin admin] PATH_INFO: `/v3/projects/8045a63f72cf457fb12a61749c9357cf/users/6fa1428fac7a4a56834f8977a41d0465/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:06.891551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc5bea-4705-4a08-81ac-f310463b91c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8045a63f72cf457fb12a61749c9357cf, user_id=6fa1428fac7a4a56834f8977a41d0465, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:06.894886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc5bea-4705-4a08-81ac-f310463b91c0 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:06.936926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 864/1701] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:06 2022] PUT /identity/v3/projects/8045a63f72cf457fb12a61749c9357cf/users/6fa1428fac7a4a56834f8977a41d0465/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:06.937931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc5bea-4705-4a08-81ac-f310463b91c0 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:07.020823 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507167a2-bbd8-451d-9e3c-de92fb27f03d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:07.102999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc5bea-4705-4a08-81ac-f310463b91c0 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:07.120072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493c48d-37d8-475d-aa15-a90e6116a155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:07.120816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493c48d-37d8-475d-aa15-a90e6116a155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:07.121308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493c48d-37d8-475d-aa15-a90e6116a155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:07.121596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493c48d-37d8-475d-aa15-a90e6116a155 admin admin] PATH_INFO: `/v3/projects/d41a39b17be64fc5afcb4ed123c6bdf6/users/484927e9ac594c40bb79637ec43f1fbe/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:07.134353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493c48d-37d8-475d-aa15-a90e6116a155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d41a39b17be64fc5afcb4ed123c6bdf6, user_id=484927e9ac594c40bb79637ec43f1fbe, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:07.137257 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493c48d-37d8-475d-aa15-a90e6116a155 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:07.156296 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 865/1702] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:06 2022] PUT /identity/v3/projects/d41a39b17be64fc5afcb4ed123c6bdf6/users/484927e9ac594c40bb79637ec43f1fbe/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:07.157674 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58065022-7beb-4e1f-b969-1b0598f47cd3 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:07.157967 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58065022-7beb-4e1f-b969-1b0598f47cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:07.158248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58065022-7beb-4e1f-b969-1b0598f47cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:07.230877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-58065022-7beb-4e1f-b969-1b0598f47cd3 None None] MFA Rules not processed for user `6fa1428fac7a4a56834f8977a41d0465`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:07.232343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-58065022-7beb-4e1f-b969-1b0598f47cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:07.241906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113b641-e392-46d7-b682-31f49b17e8ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:07.243174 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113b641-e392-46d7-b682-31f49b17e8ff service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:07.243826 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113b641-e392-46d7-b682-31f49b17e8ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:07.244487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113b641-e392-46d7-b682-31f49b17e8ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:07.248780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1113b641-e392-46d7-b682-31f49b17e8ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:07.375118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:07.384676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113b641-e392-46d7-b682-31f49b17e8ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:07.386725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113b641-e392-46d7-b682-31f49b17e8ff service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:07.396856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 866/1703] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:07.398413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0eeb7f-cdbf-40b9-a8e5-7f57e6f6ba4a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:07.398703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0eeb7f-cdbf-40b9-a8e5-7f57e6f6ba4a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:07.398976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0eeb7f-cdbf-40b9-a8e5-7f57e6f6ba4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:07.426208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 838/1704] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:07.429155 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113b641-e392-46d7-b682-31f49b17e8ff service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:07.437227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1113b641-e392-46d7-b682-31f49b17e8ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:07.471596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-3e0eeb7f-cdbf-40b9-a8e5-7f57e6f6ba4a None None] MFA Rules not processed for user `484927e9ac594c40bb79637ec43f1fbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:07.473057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3e0eeb7f-cdbf-40b9-a8e5-7f57e6f6ba4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:07.486191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113b641-e392-46d7-b682-31f49b17e8ff service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:07.504709 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:07.692446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:07.693192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:07.693473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:07.693745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] PATH_INFO: `/v3/projects/e7e1bfaf89844022baefa9fa51128e9a` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:07.696692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7e1bfaf89844022baefa9fa51128e9a)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:07.702595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:07.709391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 867/1705] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:07.712260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0eeb7f-cdbf-40b9-a8e5-7f57e6f6ba4a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:07.758576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0eeb7f-cdbf-40b9-a8e5-7f57e6f6ba4a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:07.778964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7747c31-0527-4ed8-8a5c-a5b134ea29b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:07.779847 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7747c31-0527-4ed8-8a5c-a5b134ea29b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:07.780243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7747c31-0527-4ed8-8a5c-a5b134ea29b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:07.780660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7747c31-0527-4ed8-8a5c-a5b134ea29b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:07.781395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7747c31-0527-4ed8-8a5c-a5b134ea29b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:07.790652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7747c31-0527-4ed8-8a5c-a5b134ea29b2 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:07.831559 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7e1bfaf89844022baefa9fa51128e9a', 'request_id': 'req-0af1abea-d085-4766-8f3a-b714723eec8d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:07.843361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7e1bfaf89844022baefa9fa51128e9a', 'request_id': 'req-0af1abea-d085-4766-8f3a-b714723eec8d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:07.863151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7e1bfaf89844022baefa9fa51128e9a', 'request_id': 'req-0af1abea-d085-4766-8f3a-b714723eec8d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:07.886168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 868/1706] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:07 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:07.887242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7747c31-0527-4ed8-8a5c-a5b134ea29b2 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:07.890739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e7747c31-0527-4ed8-8a5c-a5b134ea29b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:07.893799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] The token cache is being invalidate because project e7e1bfaf89844022baefa9fa51128e9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:07.894269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:07.896142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:07.899492 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 839/1707] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:07 2022] DELETE /identity/v3/projects/e7e1bfaf89844022baefa9fa51128e9a => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:07.900642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:07.903661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:08.038196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7747c31-0527-4ed8-8a5c-a5b134ea29b2 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:08.047060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626c29c-f9d1-4b9d-be31-a7cd2d3904c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:08.047699 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626c29c-f9d1-4b9d-be31-a7cd2d3904c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:08.047989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626c29c-f9d1-4b9d-be31-a7cd2d3904c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:08.048283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626c29c-f9d1-4b9d-be31-a7cd2d3904c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:08.054285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4626c29c-f9d1-4b9d-be31-a7cd2d3904c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:08.079064 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1abea-d085-4766-8f3a-b714723eec8d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:08.179854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:08.214218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626c29c-f9d1-4b9d-be31-a7cd2d3904c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:08.215735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626c29c-f9d1-4b9d-be31-a7cd2d3904c4 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:08.238911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 869/1708] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:08.240299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4410a4d-055b-4449-aeb9-ce88b7bba6c3 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:08.240593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4410a4d-055b-4449-aeb9-ce88b7bba6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:08.240869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4410a4d-055b-4449-aeb9-ce88b7bba6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:08.247275 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca825b39-a049-424f-9c70-c21ddee9b8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:08.248810 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca825b39-a049-424f-9c70-c21ddee9b8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:08.249255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca825b39-a049-424f-9c70-c21ddee9b8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:08.249622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca825b39-a049-424f-9c70-c21ddee9b8b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:08.250273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca825b39-a049-424f-9c70-c21ddee9b8b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:08.252510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca825b39-a049-424f-9c70-c21ddee9b8b1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:08.288120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-c4410a4d-055b-4449-aeb9-ce88b7bba6c3 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:08.291769 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c4410a4d-055b-4449-aeb9-ce88b7bba6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:08.299872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 840/1709] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:07 2022] POST /identity/v3/users => generated 289 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:08.308165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca825b39-a049-424f-9c70-c21ddee9b8b1 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:08.321743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 870/1710] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:08.330400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c46dfa-3abc-477b-b25d-5e941dade814 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:08.331171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c46dfa-3abc-477b-b25d-5e941dade814 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:08.331653 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c46dfa-3abc-477b-b25d-5e941dade814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:08.337859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca825b39-a049-424f-9c70-c21ddee9b8b1 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:08.352000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5e386-8bd1-4eea-b1b8-6dd43cae28f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:08.353002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5e386-8bd1-4eea-b1b8-6dd43cae28f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:08.353312 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5e386-8bd1-4eea-b1b8-6dd43cae28f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:08.353582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5e386-8bd1-4eea-b1b8-6dd43cae28f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:08.354244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5e386-8bd1-4eea-b1b8-6dd43cae28f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:08.356020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5e386-8bd1-4eea-b1b8-6dd43cae28f0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:08.385160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 841/1711] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:08.386319 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5e386-8bd1-4eea-b1b8-6dd43cae28f0 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:08.390545 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-86c46dfa-3abc-477b-b25d-5e941dade814 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:08.394191 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-86c46dfa-3abc-477b-b25d-5e941dade814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:08.411757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5e386-8bd1-4eea-b1b8-6dd43cae28f0 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:08.418279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 871/1712] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:08.419538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c46dfa-3abc-477b-b25d-5e941dade814 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:08.423350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-86c46dfa-3abc-477b-b25d-5e941dade814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:08.427236 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63ef15-583c-4d51-a385-1a920888cbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:08.427934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63ef15-583c-4d51-a385-1a920888cbe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:08.428208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63ef15-583c-4d51-a385-1a920888cbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:08.428501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63ef15-583c-4d51-a385-1a920888cbe0 admin admin] PATH_INFO: `/v3/projects/e1c3177b55f145c5bbebb2a085c44efe/users/7ea34b4276b8424c8d909b0e7331f203/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:08.452821 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c46dfa-3abc-477b-b25d-5e941dade814 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:08.465279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:08.466281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:08.466763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:08.467761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] PATH_INFO: `/v3/users/f000966ae1fa49db8d936c4d68be9340` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:08.468538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63ef15-583c-4d51-a385-1a920888cbe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c3177b55f145c5bbebb2a085c44efe, user_id=7ea34b4276b8424c8d909b0e7331f203, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:08.469416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f000966ae1fa49db8d936c4d68be9340)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:08.471856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63ef15-583c-4d51-a385-1a920888cbe0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:08.479267 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:08.505900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 842/1713] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:08 2022] PUT /identity/v3/projects/e1c3177b55f145c5bbebb2a085c44efe/users/7ea34b4276b8424c8d909b0e7331f203/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:08.505900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63ef15-583c-4d51-a385-1a920888cbe0 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:08.536204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:08.591619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f000966ae1fa49db8d936c4d68be9340'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:08.633890 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f000966ae1fa49db8d936c4d68be9340'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:08.635029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f000966ae1fa49db8d936c4d68be9340'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:08.645473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63ef15-583c-4d51-a385-1a920888cbe0 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:08.651184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f000966ae1fa49db8d936c4d68be9340'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:08.652273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f000966ae1fa49db8d936c4d68be9340'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:08.666387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56489c26-fdbe-4846-9516-890b59a8ec01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:08.667071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56489c26-fdbe-4846-9516-890b59a8ec01 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:08.667353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56489c26-fdbe-4846-9516-890b59a8ec01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:08.667634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56489c26-fdbe-4846-9516-890b59a8ec01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:08.668313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56489c26-fdbe-4846-9516-890b59a8ec01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:38:08.668610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56489c26-fdbe-4846-9516-890b59a8ec01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:08.670361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56489c26-fdbe-4846-9516-890b59a8ec01 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:08.681918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 872/1714] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:08 2022] DELETE /identity/v3/users/f000966ae1fa49db8d936c4d68be9340 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:08.682886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:08.690799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:08.710310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 843/1715] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:38:08.711785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215430e-4812-4760-a2bf-d0ab5c84b0fb None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:08.712058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215430e-4812-4760-a2bf-d0ab5c84b0fb None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:08.712333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215430e-4812-4760-a2bf-d0ab5c84b0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:08.775029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-c215430e-4812-4760-a2bf-d0ab5c84b0fb None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:08.776372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c215430e-4812-4760-a2bf-d0ab5c84b0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:08.869176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce905b-37ea-45cd-9ac2-b48645a1c6fd admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:08.883482 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e9fe-1bfe-4662-8348-ba9a5d8b95de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:08.884142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e9fe-1bfe-4662-8348-ba9a5d8b95de admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:08.884420 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e9fe-1bfe-4662-8348-ba9a5d8b95de admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:08.884692 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e9fe-1bfe-4662-8348-ba9a5d8b95de admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:08.885388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35e9fe-1bfe-4662-8348-ba9a5d8b95de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:08.891212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35e9fe-1bfe-4662-8348-ba9a5d8b95de admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:08.912882 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 873/1716] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:08.915657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e9fe-1bfe-4662-8348-ba9a5d8b95de admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:08.921309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9c35e9fe-1bfe-4662-8348-ba9a5d8b95de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:08.941229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 844/1717] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:08.941229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215430e-4812-4760-a2bf-d0ab5c84b0fb None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:08.952097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e9fe-1bfe-4662-8348-ba9a5d8b95de admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:08.960619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215430e-4812-4760-a2bf-d0ab5c84b0fb None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:08.963488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717da89-b6f4-43fa-8dd2-01a6eb0aa1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:08.964129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717da89-b6f4-43fa-8dd2-01a6eb0aa1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:08.964407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717da89-b6f4-43fa-8dd2-01a6eb0aa1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:08.966135 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717da89-b6f4-43fa-8dd2-01a6eb0aa1d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:08.966453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717da89-b6f4-43fa-8dd2-01a6eb0aa1d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:08.972722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717da89-b6f4-43fa-8dd2-01a6eb0aa1d5 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:08.975641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2dec7-cfee-4464-bf6a-0ca8b3845104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:08.976383 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2dec7-cfee-4464-bf6a-0ca8b3845104 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:08.976682 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2dec7-cfee-4464-bf6a-0ca8b3845104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:08.976952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2dec7-cfee-4464-bf6a-0ca8b3845104 admin admin] PATH_INFO: `/v3/projects/e1c3177b55f145c5bbebb2a085c44efe/users/7ea34b4276b8424c8d909b0e7331f203/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:08.979851 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2dec7-cfee-4464-bf6a-0ca8b3845104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c3177b55f145c5bbebb2a085c44efe, user_id=7ea34b4276b8424c8d909b0e7331f203, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:08.986741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2dec7-cfee-4464-bf6a-0ca8b3845104 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:09.010502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 845/1718] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:08 2022] PUT /identity/v3/projects/e1c3177b55f145c5bbebb2a085c44efe/users/7ea34b4276b8424c8d909b0e7331f203/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:09.012521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2dec7-cfee-4464-bf6a-0ca8b3845104 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:09.015338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3ee2dec7-cfee-4464-bf6a-0ca8b3845104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:09.041387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 874/1719] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:08 2022] POST /identity/v3/projects => generated 405 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:09.042955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51197a6-8103-472a-a55c-200187be5271 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:09.043150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51197a6-8103-472a-a55c-200187be5271 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:09.043425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51197a6-8103-472a-a55c-200187be5271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:09.053032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:09.156226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-d51197a6-8103-472a-a55c-200187be5271 None None] MFA Rules not processed for user `7ea34b4276b8424c8d909b0e7331f203`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:09.165895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d51197a6-8103-472a-a55c-200187be5271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:09.247085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2dec7-cfee-4464-bf6a-0ca8b3845104 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:09.452903 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:09.484054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af8271-cf14-4c8a-908c-052cdbce3c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:09.486042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af8271-cf14-4c8a-908c-052cdbce3c67 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:09.486531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af8271-cf14-4c8a-908c-052cdbce3c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:09.487025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af8271-cf14-4c8a-908c-052cdbce3c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:09.487955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 875/1720] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:09.489355 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51197a6-8103-472a-a55c-200187be5271 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:09.728008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51197a6-8103-472a-a55c-200187be5271 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:09.729152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af8271-cf14-4c8a-908c-052cdbce3c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:09.731694 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af8271-cf14-4c8a-908c-052cdbce3c67 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:09.745628 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcabcb4-12b8-4ee7-af72-0782fc40cbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:09.746444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcabcb4-12b8-4ee7-af72-0782fc40cbe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:09.746824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcabcb4-12b8-4ee7-af72-0782fc40cbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:09.747197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcabcb4-12b8-4ee7-af72-0782fc40cbe1 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:09.750583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcabcb4-12b8-4ee7-af72-0782fc40cbe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:09.754131 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcabcb4-12b8-4ee7-af72-0782fc40cbe1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:09.770267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 846/1721] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 759 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:09.771293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af8271-cf14-4c8a-908c-052cdbce3c67 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:09.778372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f4af8271-cf14-4c8a-908c-052cdbce3c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:09.819876 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 876/1722] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:09 2022] POST /identity/v3/users => generated 303 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:09.824465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcabcb4-12b8-4ee7-af72-0782fc40cbe1 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:09.842450 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcabcb4-12b8-4ee7-af72-0782fc40cbe1 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:09.856213 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414171d-dafa-4d20-a9e4-6e97472a22d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:09.856867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414171d-dafa-4d20-a9e4-6e97472a22d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:09.857312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414171d-dafa-4d20-a9e4-6e97472a22d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:09.858553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414171d-dafa-4d20-a9e4-6e97472a22d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:09.858942 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414171d-dafa-4d20-a9e4-6e97472a22d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:09.860574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414171d-dafa-4d20-a9e4-6e97472a22d6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:09.885251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 877/1723] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:38:09.889248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414171d-dafa-4d20-a9e4-6e97472a22d6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:09.916327 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414171d-dafa-4d20-a9e4-6e97472a22d6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:09.930023 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cab98b-777b-4312-9483-5d95c5512d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:09.930618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cab98b-777b-4312-9483-5d95c5512d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:09.930911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cab98b-777b-4312-9483-5d95c5512d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:09.931173 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cab98b-777b-4312-9483-5d95c5512d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:09.931829 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cab98b-777b-4312-9483-5d95c5512d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:09.933250 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af8271-cf14-4c8a-908c-052cdbce3c67 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:09.933514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cab98b-777b-4312-9483-5d95c5512d2c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:09.940199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd55aa9-9f37-4025-9461-44d968d4a732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:09.941299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd55aa9-9f37-4025-9461-44d968d4a732 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:09.941563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd55aa9-9f37-4025-9461-44d968d4a732 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:09.941839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd55aa9-9f37-4025-9461-44d968d4a732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:09.953465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 878/1724] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:38:09.957602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cab98b-777b-4312-9483-5d95c5512d2c admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:09.989176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd55aa9-9f37-4025-9461-44d968d4a732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:09.991035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd55aa9-9f37-4025-9461-44d968d4a732 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:10.001324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cab98b-777b-4312-9483-5d95c5512d2c admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:10.022678 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd4b70-52eb-43d0-871a-f2419bdf3699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:10.023329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 847/1725] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:10.024270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd4b70-52eb-43d0-871a-f2419bdf3699 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:10.025971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd4b70-52eb-43d0-871a-f2419bdf3699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:10.027457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd4b70-52eb-43d0-871a-f2419bdf3699 admin admin] PATH_INFO: `/v3/projects/94f139132b1c489d8006b88c91b754da/users/b65d9e12f6584b228134af177d536780/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:10.095923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd4b70-52eb-43d0-871a-f2419bdf3699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f139132b1c489d8006b88c91b754da, user_id=b65d9e12f6584b228134af177d536780, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:10.099632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd4b70-52eb-43d0-871a-f2419bdf3699 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:10.146508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 879/1726] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:09 2022] PUT /identity/v3/projects/94f139132b1c489d8006b88c91b754da/users/b65d9e12f6584b228134af177d536780/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:38:10.152342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed037d5-bee5-4750-8f99-7fd7c427125a None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:10.155674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed037d5-bee5-4750-8f99-7fd7c427125a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:10.157541 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed037d5-bee5-4750-8f99-7fd7c427125a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:10.225428 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-eed037d5-bee5-4750-8f99-7fd7c427125a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:10.229436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-eed037d5-bee5-4750-8f99-7fd7c427125a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:10.263274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:10.450697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 848/1727] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:10.493430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd4b70-52eb-43d0-871a-f2419bdf3699 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:10.495323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b6dd4b70-52eb-43d0-871a-f2419bdf3699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:10.751860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd4b70-52eb-43d0-871a-f2419bdf3699 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:10.792343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab11e2-1b9f-491d-b507-c2ec6a477682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:10.793964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab11e2-1b9f-491d-b507-c2ec6a477682 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:10.794872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab11e2-1b9f-491d-b507-c2ec6a477682 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:10.795939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab11e2-1b9f-491d-b507-c2ec6a477682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:10.856517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab11e2-1b9f-491d-b507-c2ec6a477682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:10.862564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab11e2-1b9f-491d-b507-c2ec6a477682 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:10.900603 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 880/1728] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:11.345617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed037d5-bee5-4750-8f99-7fd7c427125a None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:11.395561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed037d5-bee5-4750-8f99-7fd7c427125a None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:11.410744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:11.411588 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:11.411963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:11.412326 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] PATH_INFO: `/v3/users/6fa1428fac7a4a56834f8977a41d0465` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:11.415958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fa1428fac7a4a56834f8977a41d0465)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:11.423834 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:11.558045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa1428fac7a4a56834f8977a41d0465'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:11.591517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa1428fac7a4a56834f8977a41d0465'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:11.594044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa1428fac7a4a56834f8977a41d0465'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:11.595479 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa1428fac7a4a56834f8977a41d0465'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:11.617673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa1428fac7a4a56834f8977a41d0465'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:11.666486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 849/1729] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:11 2022] DELETE /identity/v3/users/6fa1428fac7a4a56834f8977a41d0465 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:11.671550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83011b-a0fe-4598-87b9-0fbd908c69bc None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:11.674192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83011b-a0fe-4598-87b9-0fbd908c69bc None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:11.674767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83011b-a0fe-4598-87b9-0fbd908c69bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:11.709594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5e83011b-a0fe-4598-87b9-0fbd908c69bc None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:11.711064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5e83011b-a0fe-4598-87b9-0fbd908c69bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:11.819043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:11.907164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:11.909362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:11.920870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 881/1730] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:12.058770 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb24733-ce85-4d4f-b2dc-a72df4f9270d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:12.069172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83011b-a0fe-4598-87b9-0fbd908c69bc None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:12.071273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5e83011b-a0fe-4598-87b9-0fbd908c69bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:12.088948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0a337-e787-4097-8ec1-d77cb305c440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:12.090821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0a337-e787-4097-8ec1-d77cb305c440 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:12.094149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0a337-e787-4097-8ec1-d77cb305c440 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:12.095948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0a337-e787-4097-8ec1-d77cb305c440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:12.132792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83011b-a0fe-4598-87b9-0fbd908c69bc None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:12.155980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:12.157204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:12.157810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:12.158204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85 admin admin] PATH_INFO: `/v3/projects/ded3c4e10415475683af7c59216d5a80` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:12.165321 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0a337-e787-4097-8ec1-d77cb305c440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:12.177024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0a337-e787-4097-8ec1-d77cb305c440 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:12.184302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ded3c4e10415475683af7c59216d5a80)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:12.194115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:12.235751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 850/1731] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:12.375793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ded3c4e10415475683af7c59216d5a80', 'request_id': 'req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:12.390671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ded3c4e10415475683af7c59216d5a80', 'request_id': 'req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:12.414715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ded3c4e10415475683af7c59216d5a80', 'request_id': 'req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:12.452700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85 admin admin] The token cache is being invalidate because project ded3c4e10415475683af7c59216d5a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:12.455162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:12.457575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8d6d2d96-24aa-41d7-b1ee-b51f9f185f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:12.463923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 882/1732] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:12 2022] DELETE /identity/v3/projects/ded3c4e10415475683af7c59216d5a80 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:12.467573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0a337-e787-4097-8ec1-d77cb305c440 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:12.501343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0a337-e787-4097-8ec1-d77cb305c440 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:12.626888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:12.707350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448354f-8c45-433e-a10e-df5481d48302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:12.710380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448354f-8c45-433e-a10e-df5481d48302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:12.712030 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448354f-8c45-433e-a10e-df5481d48302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:12.715330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448354f-8c45-433e-a10e-df5481d48302 admin admin] PATH_INFO: `/v3/users/484927e9ac594c40bb79637ec43f1fbe` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:12.718744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448354f-8c45-433e-a10e-df5481d48302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=484927e9ac594c40bb79637ec43f1fbe)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:12.725944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448354f-8c45-433e-a10e-df5481d48302 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:12.774502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a365b5-c92c-4de8-8a7f-2d83ca0c01d2 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:12.774988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a365b5-c92c-4de8-8a7f-2d83ca0c01d2 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:12.775071 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a365b5-c92c-4de8-8a7f-2d83ca0c01d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:12.872867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-a5a365b5-c92c-4de8-8a7f-2d83ca0c01d2 None None] MFA Rules not processed for user `9369796f4c5b41d6b9fe1e786fbd4b38`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:12.876504 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a5a365b5-c92c-4de8-8a7f-2d83ca0c01d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:12.936948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2448354f-8c45-433e-a10e-df5481d48302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '484927e9ac594c40bb79637ec43f1fbe'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:12.960249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2448354f-8c45-433e-a10e-df5481d48302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '484927e9ac594c40bb79637ec43f1fbe'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:12.966069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2448354f-8c45-433e-a10e-df5481d48302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '484927e9ac594c40bb79637ec43f1fbe'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:12.967984 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2448354f-8c45-433e-a10e-df5481d48302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '484927e9ac594c40bb79637ec43f1fbe'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:12.992950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2448354f-8c45-433e-a10e-df5481d48302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '484927e9ac594c40bb79637ec43f1fbe'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:13.066377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 851/1733] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:12 2022] DELETE /identity/v3/users/484927e9ac594c40bb79637ec43f1fbe => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:13.097189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 883/1734] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:13.101858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79208019-4cc9-4aae-8c04-ce48f16b0b4b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:13.102150 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79208019-4cc9-4aae-8c04-ce48f16b0b4b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:13.102424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79208019-4cc9-4aae-8c04-ce48f16b0b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:13.145539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-79208019-4cc9-4aae-8c04-ce48f16b0b4b None None] MFA Rules not processed for user `9369796f4c5b41d6b9fe1e786fbd4b38`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:13.164462 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-79208019-4cc9-4aae-8c04-ce48f16b0b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:13.282057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 852/1735] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:13.299697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a365b5-c92c-4de8-8a7f-2d83ca0c01d2 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:13.303670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a5a365b5-c92c-4de8-8a7f-2d83ca0c01d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:13.344717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6ca27-a3b0-4988-9afb-ec8086a875ca None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:13.344717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6ca27-a3b0-4988-9afb-ec8086a875ca None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:13.344959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6ca27-a3b0-4988-9afb-ec8086a875ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:13.439179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-a8f6ca27-a3b0-4988-9afb-ec8086a875ca None None] MFA Rules not processed for user `7d712e201f2c47a8a48b6e24559e0d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:13.440816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a8f6ca27-a3b0-4988-9afb-ec8086a875ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:13.471668 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:13.529386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a365b5-c92c-4de8-8a7f-2d83ca0c01d2 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:13.594963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:13.759225 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 853/1736] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:13.764857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c439e0-5695-46a9-ab53-7e59ab70c76b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:13.767249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c439e0-5695-46a9-ab53-7e59ab70c76b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:13.768955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c439e0-5695-46a9-ab53-7e59ab70c76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:13.902666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-56c439e0-5695-46a9-ab53-7e59ab70c76b None None] MFA Rules not processed for user `fe0fcdeafae645a582afcee3dde79ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:13.905345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842d30b-57f2-4042-94d0-388848f40051 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:13.906068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-56c439e0-5695-46a9-ab53-7e59ab70c76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:13.907442 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842d30b-57f2-4042-94d0-388848f40051 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:13.907809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842d30b-57f2-4042-94d0-388848f40051 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:13.908643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842d30b-57f2-4042-94d0-388848f40051 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:13.957139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842d30b-57f2-4042-94d0-388848f40051 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:13.963985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842d30b-57f2-4042-94d0-388848f40051 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:13.992001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 884/1737] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:14.019938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c88a6-9416-4c63-aa3b-836049f72a7e None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:14.020340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c88a6-9416-4c63-aa3b-836049f72a7e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:14.020604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c88a6-9416-4c63-aa3b-836049f72a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:14.092392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-1d6c88a6-9416-4c63-aa3b-836049f72a7e None None] MFA Rules not processed for user `7d712e201f2c47a8a48b6e24559e0d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:14.094469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 854/1738] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:14.096865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c439e0-5695-46a9-ab53-7e59ab70c76b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:14.098956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1d6c88a6-9416-4c63-aa3b-836049f72a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:14.099822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-56c439e0-5695-46a9-ab53-7e59ab70c76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:14.124488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 885/1739] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:14.124488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c88a6-9416-4c63-aa3b-836049f72a7e None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:14.124897 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1d6c88a6-9416-4c63-aa3b-836049f72a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:14.161170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c88a6-9416-4c63-aa3b-836049f72a7e None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:14.175001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d52847-8a0d-4916-8dd5-f221084caa3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:14.176758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d52847-8a0d-4916-8dd5-f221084caa3e service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:14.178593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d52847-8a0d-4916-8dd5-f221084caa3e service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:14.178990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d52847-8a0d-4916-8dd5-f221084caa3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:14.197942 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d52847-8a0d-4916-8dd5-f221084caa3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:14.199771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d52847-8a0d-4916-8dd5-f221084caa3e service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:14.222286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 886/1740] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:14.246951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c439e0-5695-46a9-ab53-7e59ab70c76b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:14.252460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:14.253439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:14.253864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:14.254246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] PATH_INFO: `/v3/projects/8045a63f72cf457fb12a61749c9357cf` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:14.255637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8045a63f72cf457fb12a61749c9357cf)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:14.257894 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:14.349260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8045a63f72cf457fb12a61749c9357cf', 'request_id': 'req-408dfd7e-7f05-4a59-86ff-df22ba649336'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.360924 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d52847-8a0d-4916-8dd5-f221084caa3e service nova] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:14.362717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a2d52847-8a0d-4916-8dd5-f221084caa3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:14.366628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8045a63f72cf457fb12a61749c9357cf', 'request_id': 'req-408dfd7e-7f05-4a59-86ff-df22ba649336'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.382497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8045a63f72cf457fb12a61749c9357cf', 'request_id': 'req-408dfd7e-7f05-4a59-86ff-df22ba649336'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.387727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d52847-8a0d-4916-8dd5-f221084caa3e service nova] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:14.418968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] The token cache is being invalidate because project 8045a63f72cf457fb12a61749c9357cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:14.419391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.421028 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.425325 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 855/1741] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:14 2022] DELETE /identity/v3/projects/8045a63f72cf457fb12a61749c9357cf => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:14.429522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:14.432214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:14.456714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:14.557029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:14.557723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:14.558002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:14.558278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] PATH_INFO: `/v3/projects/d41a39b17be64fc5afcb4ed123c6bdf6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:14.574245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d41a39b17be64fc5afcb4ed123c6bdf6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:14.576183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:14.669966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408dfd7e-7f05-4a59-86ff-df22ba649336 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:14.684150 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faff340-0a1a-40b4-9441-59b929c16988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:14.688611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faff340-0a1a-40b4-9441-59b929c16988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:14.688611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faff340-0a1a-40b4-9441-59b929c16988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:14.688611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faff340-0a1a-40b4-9441-59b929c16988 admin admin] PATH_INFO: `/v3/users/7ea34b4276b8424c8d909b0e7331f203` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:14.693994 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faff340-0a1a-40b4-9441-59b929c16988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea34b4276b8424c8d909b0e7331f203)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:14.710518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faff340-0a1a-40b4-9441-59b929c16988 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:14.746160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd41a39b17be64fc5afcb4ed123c6bdf6', 'request_id': 'req-b2c85ea7-01c3-4004-97ee-53372d494929'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.759664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd41a39b17be64fc5afcb4ed123c6bdf6', 'request_id': 'req-b2c85ea7-01c3-4004-97ee-53372d494929'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.777963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd41a39b17be64fc5afcb4ed123c6bdf6', 'request_id': 'req-b2c85ea7-01c3-4004-97ee-53372d494929'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.780838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:14.804521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] The token cache is being invalidate because project d41a39b17be64fc5afcb4ed123c6bdf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:14.806002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.807673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.811503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 887/1742] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:14 2022] DELETE /identity/v3/projects/d41a39b17be64fc5afcb4ed123c6bdf6 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:38:14.813530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:14.815869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:14.863766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8faff340-0a1a-40b4-9441-59b929c16988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea34b4276b8424c8d909b0e7331f203'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.886302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8faff340-0a1a-40b4-9441-59b929c16988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea34b4276b8424c8d909b0e7331f203'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.890718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8faff340-0a1a-40b4-9441-59b929c16988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea34b4276b8424c8d909b0e7331f203'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.893868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8faff340-0a1a-40b4-9441-59b929c16988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea34b4276b8424c8d909b0e7331f203'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.916204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8faff340-0a1a-40b4-9441-59b929c16988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea34b4276b8424c8d909b0e7331f203'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:14.997250 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 856/1743] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:14 2022] DELETE /identity/v3/users/7ea34b4276b8424c8d909b0e7331f203 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:14.999124 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f259a-cc37-4fdd-9df8-f3c4a5674176 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:14.999566 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f259a-cc37-4fdd-9df8-f3c4a5674176 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:14.999873 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f259a-cc37-4fdd-9df8-f3c4a5674176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:15.055827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c85ea7-01c3-4004-97ee-53372d494929 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:15.072607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-098f259a-cc37-4fdd-9df8-f3c4a5674176 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:15.073939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-098f259a-cc37-4fdd-9df8-f3c4a5674176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:15.247351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea7750-b6d5-42db-92bd-86d755f450aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:15.248036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea7750-b6d5-42db-92bd-86d755f450aa service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:15.248329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea7750-b6d5-42db-92bd-86d755f450aa service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:15.248615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea7750-b6d5-42db-92bd-86d755f450aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:15.255291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3aea7750-b6d5-42db-92bd-86d755f450aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:15.267747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 857/1744] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:15.270148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80ff0c-6f2f-4834-82dc-fa6823b00480 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:15.271251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80ff0c-6f2f-4834-82dc-fa6823b00480 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:15.271689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80ff0c-6f2f-4834-82dc-fa6823b00480 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:15.274512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 858/1745] 10.222.0.52 () {54 vars in 912 bytes} [Wed Sep 28 08:38:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:38:15.276081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d660d-2f18-4b8c-8e8e-d48ef1272e39 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:15.277123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d660d-2f18-4b8c-8e8e-d48ef1272e39 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:15.277643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d660d-2f18-4b8c-8e8e-d48ef1272e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:15.403567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-a28d660d-2f18-4b8c-8e8e-d48ef1272e39 None None] MFA Rules not processed for user `b65d9e12f6584b228134af177d536780`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:15.405496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a28d660d-2f18-4b8c-8e8e-d48ef1272e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:15.411345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea7750-b6d5-42db-92bd-86d755f450aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:15.419571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea7750-b6d5-42db-92bd-86d755f450aa service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:15.439112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:15.458062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 888/1746] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:15.463471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc39a3b-1a29-4752-b774-4a2f56f32e3a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:15.464919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc39a3b-1a29-4752-b774-4a2f56f32e3a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:15.465463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc39a3b-1a29-4752-b774-4a2f56f32e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:15.532363 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-fbc39a3b-1a29-4752-b774-4a2f56f32e3a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:15.537882 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-fbc39a3b-1a29-4752-b774-4a2f56f32e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:15.562867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 889/1747] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:15.564572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7b930-54aa-4750-9454-fa56a57c221f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:15.565831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7b930-54aa-4750-9454-fa56a57c221f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:15.566243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7b930-54aa-4750-9454-fa56a57c221f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:15.615478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 859/1748] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:15 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:15.615478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d660d-2f18-4b8c-8e8e-d48ef1272e39 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:15.641007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d660d-2f18-4b8c-8e8e-d48ef1272e39 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:15.662107 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1f7dd-0951-48b9-af58-040edb8f6be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:15.663232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1f7dd-0951-48b9-af58-040edb8f6be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:15.663976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1f7dd-0951-48b9-af58-040edb8f6be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:15.664482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1f7dd-0951-48b9-af58-040edb8f6be1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:15.665778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1f7dd-0951-48b9-af58-040edb8f6be1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:38:15.666343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1f7dd-0951-48b9-af58-040edb8f6be1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:15.669013 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1f7dd-0951-48b9-af58-040edb8f6be1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:15.690337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:15.695603 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f6f7b930-54aa-4750-9454-fa56a57c221f None None] MFA Rules not processed for user `736efafe9fc94b45bc75566b1696c498`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:15.700730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f6f7b930-54aa-4750-9454-fa56a57c221f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:15.714987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 860/1749] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:38:15.716124 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1f7dd-0951-48b9-af58-040edb8f6be1 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:15.723671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-78d1f7dd-0951-48b9-af58-040edb8f6be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:15.767920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1f7dd-0951-48b9-af58-040edb8f6be1 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:15.782537 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dae35-01d0-4398-8314-22127d2fcab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:15.783316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dae35-01d0-4398-8314-22127d2fcab9 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:15.783609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dae35-01d0-4398-8314-22127d2fcab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:15.783886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dae35-01d0-4398-8314-22127d2fcab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:15.784626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dae35-01d0-4398-8314-22127d2fcab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:15.789487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dae35-01d0-4398-8314-22127d2fcab9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:15.876669 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 861/1750] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:15 2022] POST /identity/v3/projects => generated 403 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:15.876669 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dae35-01d0-4398-8314-22127d2fcab9 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:15.881516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 890/1751] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:38:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:15.882612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7b930-54aa-4750-9454-fa56a57c221f None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:15.912675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7b930-54aa-4750-9454-fa56a57c221f None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:16.089983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb7378-4e13-4770-ba3f-700b39d82e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:16.090780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb7378-4e13-4770-ba3f-700b39d82e73 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:16.092279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb7378-4e13-4770-ba3f-700b39d82e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:16.092595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb7378-4e13-4770-ba3f-700b39d82e73 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:16.095591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb7378-4e13-4770-ba3f-700b39d82e73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:16.100844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb7378-4e13-4770-ba3f-700b39d82e73 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:16.170595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 891/1752] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:15 2022] POST /identity/v3/users => generated 308 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:16.170595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb7378-4e13-4770-ba3f-700b39d82e73 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:16.180246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dae35-01d0-4398-8314-22127d2fcab9 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:16.189003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-6deb7378-4e13-4770-ba3f-700b39d82e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:16.218514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbf009-804d-440a-bec1-d1ab523d6f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:16.228978 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbf009-804d-440a-bec1-d1ab523d6f1b admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:16.234000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbf009-804d-440a-bec1-d1ab523d6f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:16.234439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbf009-804d-440a-bec1-d1ab523d6f1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:16.235704 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbf009-804d-440a-bec1-d1ab523d6f1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:16.251921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbf009-804d-440a-bec1-d1ab523d6f1b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:16.350558 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb7378-4e13-4770-ba3f-700b39d82e73 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:16.356450 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01eb7f-c8f5-4d96-baef-c99fd75928bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:16.358857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01eb7f-c8f5-4d96-baef-c99fd75928bc service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:16.359128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01eb7f-c8f5-4d96-baef-c99fd75928bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:16.359453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01eb7f-c8f5-4d96-baef-c99fd75928bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:16.375726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 862/1753] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:15 2022] POST /identity/v3/projects => generated 391 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:16.376924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbf009-804d-440a-bec1-d1ab523d6f1b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:16.381759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01eb7f-c8f5-4d96-baef-c99fd75928bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:16.383228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01eb7f-c8f5-4d96-baef-c99fd75928bc service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:16.406362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 892/1754] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:16.408663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c232aa-78d0-46be-a0a9-fb9b48dfc143 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:16.408956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c232aa-78d0-46be-a0a9-fb9b48dfc143 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:16.409263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c232aa-78d0-46be-a0a9-fb9b48dfc143 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:16.417846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 893/1755] 10.222.0.52 () {54 vars in 912 bytes} [Wed Sep 28 08:38:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:38:16.418807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c232aa-78d0-46be-a0a9-fb9b48dfc143 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:16.422780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-24c232aa-78d0-46be-a0a9-fb9b48dfc143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:16.563374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbf009-804d-440a-bec1-d1ab523d6f1b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:16.580448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5012e73-dbad-4713-9078-5920a42e3c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:16.582642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5012e73-dbad-4713-9078-5920a42e3c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:16.583961 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5012e73-dbad-4713-9078-5920a42e3c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:16.584767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5012e73-dbad-4713-9078-5920a42e3c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:16.589416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5012e73-dbad-4713-9078-5920a42e3c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:16.592374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5012e73-dbad-4713-9078-5920a42e3c08 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:16.618977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:16.638069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 863/1756] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:16.640376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5012e73-dbad-4713-9078-5920a42e3c08 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:16.647702 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:16.664288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5012e73-dbad-4713-9078-5920a42e3c08 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:16.679291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63bdaa-57db-4d06-98aa-675ecb6b11ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:16.679937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63bdaa-57db-4d06-98aa-675ecb6b11ee admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:16.680221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63bdaa-57db-4d06-98aa-675ecb6b11ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:16.680500 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63bdaa-57db-4d06-98aa-675ecb6b11ee admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:16.688793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63bdaa-57db-4d06-98aa-675ecb6b11ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:16.691327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63bdaa-57db-4d06-98aa-675ecb6b11ee admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:16.739371 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c232aa-78d0-46be-a0a9-fb9b48dfc143 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:16.767219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 864/1757] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:16 2022] POST /identity/v3/users => generated 296 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:16.775064 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6c29e-55c7-45d8-8d64-ab761cc76d36 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:16.777230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6c29e-55c7-45d8-8d64-ab761cc76d36 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:16.778239 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:16.778871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6c29e-55c7-45d8-8d64-ab761cc76d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:16.783028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:16.784013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:16.784933 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] PATH_INFO: `/v3/projects/e1c3177b55f145c5bbebb2a085c44efe` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:16.792357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1c3177b55f145c5bbebb2a085c44efe)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:16.802068 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:16.878304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-c3f6c29e-55c7-45d8-8d64-ab761cc76d36 None None] MFA Rules not processed for user `736efafe9fc94b45bc75566b1696c498`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:16.879314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c3f6c29e-55c7-45d8-8d64-ab761cc76d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:17.035875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1c3177b55f145c5bbebb2a085c44efe', 'request_id': 'req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:17.080265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 865/1758] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:38:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:17.081345 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6c29e-55c7-45d8-8d64-ab761cc76d36 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:17.110834 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6c29e-55c7-45d8-8d64-ab761cc76d36 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:17.113035 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1c3177b55f145c5bbebb2a085c44efe', 'request_id': 'req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:17.137130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1c3177b55f145c5bbebb2a085c44efe', 'request_id': 'req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:17.178438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] The token cache is being invalidate because project e1c3177b55f145c5bbebb2a085c44efe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:17.179170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:17.181516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:17.188362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 894/1759] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:16 2022] DELETE /identity/v3/projects/e1c3177b55f145c5bbebb2a085c44efe => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:17.189995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:17.192327 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:17.302491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443787e4-490b-4162-b67c-059e57f4de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:17.302491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443787e4-490b-4162-b67c-059e57f4de14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:17.302896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443787e4-490b-4162-b67c-059e57f4de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:17.303037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443787e4-490b-4162-b67c-059e57f4de14 admin admin] PATH_INFO: `/v3/projects/1cf0587f5b0146738736237e64f33d66/users/0aa8933ba0524fea94470931850a35e6/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:17.345409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443787e4-490b-4162-b67c-059e57f4de14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf0587f5b0146738736237e64f33d66, user_id=0aa8933ba0524fea94470931850a35e6, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:17.348644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443787e4-490b-4162-b67c-059e57f4de14 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:17.366204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d1941-b4fe-48a9-93b7-f7a3573e16f7 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:17.377359 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 866/1760] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:17 2022] PUT /identity/v3/projects/1cf0587f5b0146738736237e64f33d66/users/0aa8933ba0524fea94470931850a35e6/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:17.378530 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443787e4-490b-4162-b67c-059e57f4de14 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:17.386345 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-443787e4-490b-4162-b67c-059e57f4de14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:17.525941 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed984f3-76ab-40ee-bd0a-a27fc2a2ba28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:17.526917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed984f3-76ab-40ee-bd0a-a27fc2a2ba28 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:17.527364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed984f3-76ab-40ee-bd0a-a27fc2a2ba28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:17.527795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed984f3-76ab-40ee-bd0a-a27fc2a2ba28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:17.528647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed984f3-76ab-40ee-bd0a-a27fc2a2ba28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:17.530990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed984f3-76ab-40ee-bd0a-a27fc2a2ba28 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:17.572161 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 895/1761] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:17.574538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443787e4-490b-4162-b67c-059e57f4de14 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:17.575219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1496fe1-5cb5-4f34-9629-e44315246912 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:17.575525 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1496fe1-5cb5-4f34-9629-e44315246912 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:17.575807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1496fe1-5cb5-4f34-9629-e44315246912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:17.590773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40185ef-64da-4875-adac-cb46500ac881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:17.593433 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40185ef-64da-4875-adac-cb46500ac881 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:17.594558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40185ef-64da-4875-adac-cb46500ac881 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:17.595055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40185ef-64da-4875-adac-cb46500ac881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:17.608303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d40185ef-64da-4875-adac-cb46500ac881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:17.639389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40185ef-64da-4875-adac-cb46500ac881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:17.644722 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40185ef-64da-4875-adac-cb46500ac881 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:17.651522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f1496fe1-5cb5-4f34-9629-e44315246912 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:17.652961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f1496fe1-5cb5-4f34-9629-e44315246912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:17.666805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 867/1762] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:17.667857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40185ef-64da-4875-adac-cb46500ac881 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:17.675336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d40185ef-64da-4875-adac-cb46500ac881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:17.682538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 896/1763] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:17.684617 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074fed8-999e-424d-acc2-d34c476d2f08 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:17.685048 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074fed8-999e-424d-acc2-d34c476d2f08 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:17.685981 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074fed8-999e-424d-acc2-d34c476d2f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:17.751546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40185ef-64da-4875-adac-cb46500ac881 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:17.767437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7387acb-e7b3-43e6-82d9-a15010c71d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:17.768902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7387acb-e7b3-43e6-82d9-a15010c71d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:17.769408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7387acb-e7b3-43e6-82d9-a15010c71d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:17.769871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7387acb-e7b3-43e6-82d9-a15010c71d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:17.770824 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7387acb-e7b3-43e6-82d9-a15010c71d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:17.772719 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7387acb-e7b3-43e6-82d9-a15010c71d24 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:17.787484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e074fed8-999e-424d-acc2-d34c476d2f08 None None] MFA Rules not processed for user `7e3b9ceb95374f1dbf356122c8ed463c`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:17.788990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e074fed8-999e-424d-acc2-d34c476d2f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:17.800679 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:17.825474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 868/1764] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:17.828620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7387acb-e7b3-43e6-82d9-a15010c71d24 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:17.860758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7387acb-e7b3-43e6-82d9-a15010c71d24 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:17.872595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1a664-fe3b-46ce-a0c7-8dfee32fdd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:17.874040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1a664-fe3b-46ce-a0c7-8dfee32fdd71 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:17.883541 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1a664-fe3b-46ce-a0c7-8dfee32fdd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:17.883977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1a664-fe3b-46ce-a0c7-8dfee32fdd71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:17.884929 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1a664-fe3b-46ce-a0c7-8dfee32fdd71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:17.887330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1a664-fe3b-46ce-a0c7-8dfee32fdd71 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:17.900303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:17.923014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 869/1765] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:17.924213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac8d0-ba0d-44d1-bf3c-cac982fa6452 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:17.924506 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac8d0-ba0d-44d1-bf3c-cac982fa6452 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:17.924789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac8d0-ba0d-44d1-bf3c-cac982fa6452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:17.982439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-71aac8d0-ba0d-44d1-bf3c-cac982fa6452 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:17.983840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-71aac8d0-ba0d-44d1-bf3c-cac982fa6452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:17.995328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 870/1766] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:17.999874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aac8d0-ba0d-44d1-bf3c-cac982fa6452 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:18.015178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 897/1767] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:18.024973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aac8d0-ba0d-44d1-bf3c-cac982fa6452 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:18.026523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074fed8-999e-424d-acc2-d34c476d2f08 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:18.029024 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e074fed8-999e-424d-acc2-d34c476d2f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:18.043210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442437d-b37d-4f69-8fa3-5a5ffb4c6120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:18.044093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442437d-b37d-4f69-8fa3-5a5ffb4c6120 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:18.044552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442437d-b37d-4f69-8fa3-5a5ffb4c6120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:18.044870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442437d-b37d-4f69-8fa3-5a5ffb4c6120 admin admin] PATH_INFO: `/v3/projects/1cf0587f5b0146738736237e64f33d66/users/0aa8933ba0524fea94470931850a35e6/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:18.048087 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442437d-b37d-4f69-8fa3-5a5ffb4c6120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf0587f5b0146738736237e64f33d66, user_id=0aa8933ba0524fea94470931850a35e6, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:18.051314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442437d-b37d-4f69-8fa3-5a5ffb4c6120 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:18.079551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 871/1768] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:17 2022] PUT /identity/v3/projects/1cf0587f5b0146738736237e64f33d66/users/0aa8933ba0524fea94470931850a35e6/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:18.083576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442437d-b37d-4f69-8fa3-5a5ffb4c6120 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:18.181907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074fed8-999e-424d-acc2-d34c476d2f08 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:18.272958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442437d-b37d-4f69-8fa3-5a5ffb4c6120 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:18.283484 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83de539-5439-4fc0-a40f-4ab99066c997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:18.284218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83de539-5439-4fc0-a40f-4ab99066c997 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:18.287545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83de539-5439-4fc0-a40f-4ab99066c997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:18.287957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83de539-5439-4fc0-a40f-4ab99066c997 admin admin] PATH_INFO: `/v3/projects/52fe6054bbd34fb5b302a30c7b9e6c3d/users/bf69342d99824c90bce94e7e47ebd48a/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:18.322828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83de539-5439-4fc0-a40f-4ab99066c997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52fe6054bbd34fb5b302a30c7b9e6c3d, user_id=bf69342d99824c90bce94e7e47ebd48a, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:18.327719 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83de539-5439-4fc0-a40f-4ab99066c997 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:18.358679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 872/1769] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:18 2022] PUT /identity/v3/projects/52fe6054bbd34fb5b302a30c7b9e6c3d/users/bf69342d99824c90bce94e7e47ebd48a/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:18.363994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f3b3b-a14f-4233-9792-dd80ef3f7bff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:18.364789 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f3b3b-a14f-4233-9792-dd80ef3f7bff service placement] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:18.365214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f3b3b-a14f-4233-9792-dd80ef3f7bff service placement] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:18.365597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f3b3b-a14f-4233-9792-dd80ef3f7bff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:18.375709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83de539-5439-4fc0-a40f-4ab99066c997 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:18.376296 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-404f3b3b-a14f-4233-9792-dd80ef3f7bff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:18.411219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f3b3b-a14f-4233-9792-dd80ef3f7bff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:18.419555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f3b3b-a14f-4233-9792-dd80ef3f7bff service placement] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:18.478407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 898/1770] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:18.479469 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f3b3b-a14f-4233-9792-dd80ef3f7bff service placement] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:18.487517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-404f3b3b-a14f-4233-9792-dd80ef3f7bff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:18.599828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:18.624463 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83de539-5439-4fc0-a40f-4ab99066c997 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:18.653163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c8639-0ff6-4e7f-8ab8-c73909ae1dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:18.653163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c8639-0ff6-4e7f-8ab8-c73909ae1dfd admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:18.653163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c8639-0ff6-4e7f-8ab8-c73909ae1dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:18.653163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c8639-0ff6-4e7f-8ab8-c73909ae1dfd admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:18.662550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c8639-0ff6-4e7f-8ab8-c73909ae1dfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:38:18.664742 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c8639-0ff6-4e7f-8ab8-c73909ae1dfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:18.674474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c8639-0ff6-4e7f-8ab8-c73909ae1dfd admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:18.723615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 873/1771] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:38:18.724987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e9291-4401-4f7c-847f-81d95a8aa3ab None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:18.731865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e9291-4401-4f7c-847f-81d95a8aa3ab None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:18.732271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e9291-4401-4f7c-847f-81d95a8aa3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:18.732695 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f3b3b-a14f-4233-9792-dd80ef3f7bff service placement] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:18.860664 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-bc1e9291-4401-4f7c-847f-81d95a8aa3ab None None] MFA Rules not processed for user `bf69342d99824c90bce94e7e47ebd48a`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:18.861981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-bc1e9291-4401-4f7c-847f-81d95a8aa3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:18.912758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd849a-f116-4f01-8f0d-fb310f733ce2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:18.913885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd849a-f116-4f01-8f0d-fb310f733ce2 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:18.914273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd849a-f116-4f01-8f0d-fb310f733ce2 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:18.914651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd849a-f116-4f01-8f0d-fb310f733ce2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:19.048978 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:19.120521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd849a-f116-4f01-8f0d-fb310f733ce2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:19.125367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd849a-f116-4f01-8f0d-fb310f733ce2 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:19.130329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 874/1772] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:19.140867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e9291-4401-4f7c-847f-81d95a8aa3ab None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:19.175581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 899/1773] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:19.182353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd849a-f116-4f01-8f0d-fb310f733ce2 service nova] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:19.192385 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e9291-4401-4f7c-847f-81d95a8aa3ab None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:19.230214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cc4c8-b1ca-4353-b354-5ba6a73bf125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:19.231769 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cc4c8-b1ca-4353-b354-5ba6a73bf125 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:19.234858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cc4c8-b1ca-4353-b354-5ba6a73bf125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:19.235730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cc4c8-b1ca-4353-b354-5ba6a73bf125 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:19.236880 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cc4c8-b1ca-4353-b354-5ba6a73bf125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:19.244513 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd849a-f116-4f01-8f0d-fb310f733ce2 service nova] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:19.251440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cc4c8-b1ca-4353-b354-5ba6a73bf125 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:19.270068 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1f05d-ac30-4bff-8c1a-8757178c33b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:19.271414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1f05d-ac30-4bff-8c1a-8757178c33b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:19.273067 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1f05d-ac30-4bff-8c1a-8757178c33b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:19.273494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1f05d-ac30-4bff-8c1a-8757178c33b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:19.274765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c1f05d-ac30-4bff-8c1a-8757178c33b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:19.278657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c1f05d-ac30-4bff-8c1a-8757178c33b2 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:19.345858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 875/1774] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:19 2022] POST /identity/v3/projects => generated 365 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:19.352502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cc4c8-b1ca-4353-b354-5ba6a73bf125 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:19.387060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 900/1775] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:19 2022] POST /identity/v3/projects => generated 393 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:19.388074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1f05d-ac30-4bff-8c1a-8757178c33b2 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:19.691573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:19.695837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cc4c8-b1ca-4353-b354-5ba6a73bf125 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:19.716272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692afecc-6343-4d20-a45a-9a25b6445807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:19.718866 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692afecc-6343-4d20-a45a-9a25b6445807 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:19.719130 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692afecc-6343-4d20-a45a-9a25b6445807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:19.719408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692afecc-6343-4d20-a45a-9a25b6445807 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:19.720075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692afecc-6343-4d20-a45a-9a25b6445807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:19.723279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1f05d-ac30-4bff-8c1a-8757178c33b2 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:19.729302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692afecc-6343-4d20-a45a-9a25b6445807 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:19.744492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371b844-a458-4733-b470-968b46f11930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:19.745956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371b844-a458-4733-b470-968b46f11930 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:19.746245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371b844-a458-4733-b470-968b46f11930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:19.746518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371b844-a458-4733-b470-968b46f11930 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:19.747187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371b844-a458-4733-b470-968b46f11930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:19.762820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371b844-a458-4733-b470-968b46f11930 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:19.806814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 876/1776] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:19 2022] POST /identity/v3/users => generated 283 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:19.811852 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692afecc-6343-4d20-a45a-9a25b6445807 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:19.861016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692afecc-6343-4d20-a45a-9a25b6445807 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:19.862220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 901/1777] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:19 2022] POST /identity/v3/users => generated 303 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:19.869513 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371b844-a458-4733-b470-968b46f11930 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:19.900496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509326e-ff32-4482-bf45-28a1a551d396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:19.901157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509326e-ff32-4482-bf45-28a1a551d396 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:19.901443 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509326e-ff32-4482-bf45-28a1a551d396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:19.904803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509326e-ff32-4482-bf45-28a1a551d396 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:19.906245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509326e-ff32-4482-bf45-28a1a551d396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:19.916862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371b844-a458-4733-b470-968b46f11930 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:19.922720 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509326e-ff32-4482-bf45-28a1a551d396 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:19.942131 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f62ba-a968-4571-8c44-dbd391c41902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:19.944301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f62ba-a968-4571-8c44-dbd391c41902 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:19.944880 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f62ba-a968-4571-8c44-dbd391c41902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:19.947055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f62ba-a968-4571-8c44-dbd391c41902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:19.948764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f62ba-a968-4571-8c44-dbd391c41902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:19.955067 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f62ba-a968-4571-8c44-dbd391c41902 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:19.964789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 877/1778] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:19.970266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509326e-ff32-4482-bf45-28a1a551d396 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:19.989178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 902/1779] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:19.990562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f62ba-a968-4571-8c44-dbd391c41902 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:19.996598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509326e-ff32-4482-bf45-28a1a551d396 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:20.007175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6eeb30-1d9c-4cb6-8a09-718b6cd8b45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:20.007814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6eeb30-1d9c-4cb6-8a09-718b6cd8b45d admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:20.008094 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6eeb30-1d9c-4cb6-8a09-718b6cd8b45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:20.008365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6eeb30-1d9c-4cb6-8a09-718b6cd8b45d admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:20.010339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6eeb30-1d9c-4cb6-8a09-718b6cd8b45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:20.014567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6eeb30-1d9c-4cb6-8a09-718b6cd8b45d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:20.027081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f62ba-a968-4571-8c44-dbd391c41902 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:20.045471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 878/1780] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:20.046604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6eeb30-1d9c-4cb6-8a09-718b6cd8b45d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:20.047777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f36158-9de7-4558-bd0d-d52a0bb88c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:20.050677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f36158-9de7-4558-bd0d-d52a0bb88c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:20.051103 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f36158-9de7-4558-bd0d-d52a0bb88c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:20.052512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f36158-9de7-4558-bd0d-d52a0bb88c09 admin admin] PATH_INFO: `/v3/projects/9e2d753c4cff4ded80d6c2345af8cbc8/users/084a0502f701456c9c188833f0ca1239/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:20.078678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6eeb30-1d9c-4cb6-8a09-718b6cd8b45d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:20.098949 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abb329-fa84-4dc8-94f7-57f16f401e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:20.102802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abb329-fa84-4dc8-94f7-57f16f401e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:20.103151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abb329-fa84-4dc8-94f7-57f16f401e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:20.103356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abb329-fa84-4dc8-94f7-57f16f401e8d admin admin] PATH_INFO: `/v3/projects/2f615366bc9941e9abc511a35f389344/users/832fec615e5149e4bcaa807b7a1813df/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:20.169245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f36158-9de7-4558-bd0d-d52a0bb88c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e2d753c4cff4ded80d6c2345af8cbc8, user_id=084a0502f701456c9c188833f0ca1239, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:20.177433 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abb329-fa84-4dc8-94f7-57f16f401e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f615366bc9941e9abc511a35f389344, user_id=832fec615e5149e4bcaa807b7a1813df, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:20.177691 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f36158-9de7-4558-bd0d-d52a0bb88c09 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:20.186930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abb329-fa84-4dc8-94f7-57f16f401e8d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:20.217258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 903/1781] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:19 2022] PUT /identity/v3/projects/9e2d753c4cff4ded80d6c2345af8cbc8/users/084a0502f701456c9c188833f0ca1239/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:20.218795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f36158-9de7-4558-bd0d-d52a0bb88c09 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:20.246822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 879/1782] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:20 2022] PUT /identity/v3/projects/2f615366bc9941e9abc511a35f389344/users/832fec615e5149e4bcaa807b7a1813df/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:20.252720 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb104e7-3aae-4acb-90d9-2177e43fba14 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:20.254750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb104e7-3aae-4acb-90d9-2177e43fba14 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:20.254941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb104e7-3aae-4acb-90d9-2177e43fba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:20.361816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-2bb104e7-3aae-4acb-90d9-2177e43fba14 None None] MFA Rules not processed for user `832fec615e5149e4bcaa807b7a1813df`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:20.363698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2bb104e7-3aae-4acb-90d9-2177e43fba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:20.546272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f36158-9de7-4558-bd0d-d52a0bb88c09 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:20.605777 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:20.667534 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 880/1783] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:20.678748 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb104e7-3aae-4acb-90d9-2177e43fba14 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:20.795375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d8e6e-c65a-4cbe-a7a1-229c4c1f0685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:20.796030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d8e6e-c65a-4cbe-a7a1-229c4c1f0685 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:20.796492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d8e6e-c65a-4cbe-a7a1-229c4c1f0685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:20.796775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d8e6e-c65a-4cbe-a7a1-229c4c1f0685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:20.797744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d8e6e-c65a-4cbe-a7a1-229c4c1f0685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:20.800774 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d8e6e-c65a-4cbe-a7a1-229c4c1f0685 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:20.831631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 904/1784] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 613 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:20.835654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d8e6e-c65a-4cbe-a7a1-229c4c1f0685 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:20.863794 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d8e6e-c65a-4cbe-a7a1-229c4c1f0685 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:20.882184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd052cc4-644e-4db8-a313-f66ee56a160a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:20.882890 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb104e7-3aae-4acb-90d9-2177e43fba14 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:20.883550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd052cc4-644e-4db8-a313-f66ee56a160a admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:20.883996 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd052cc4-644e-4db8-a313-f66ee56a160a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:20.884421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd052cc4-644e-4db8-a313-f66ee56a160a admin admin] PATH_INFO: `/v3/projects/9e2d753c4cff4ded80d6c2345af8cbc8/users/084a0502f701456c9c188833f0ca1239/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:20.889716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd052cc4-644e-4db8-a313-f66ee56a160a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e2d753c4cff4ded80d6c2345af8cbc8, user_id=084a0502f701456c9c188833f0ca1239, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:20.896678 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd052cc4-644e-4db8-a313-f66ee56a160a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:20.897583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76f35c-d4e9-4d79-89c5-2534f3890945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:20.898323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76f35c-d4e9-4d79-89c5-2534f3890945 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:20.898708 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76f35c-d4e9-4d79-89c5-2534f3890945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:20.899084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76f35c-d4e9-4d79-89c5-2534f3890945 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:20.900098 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76f35c-d4e9-4d79-89c5-2534f3890945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:20.902214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76f35c-d4e9-4d79-89c5-2534f3890945 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:20.939981 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 905/1785] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:20 2022] PUT /identity/v3/projects/9e2d753c4cff4ded80d6c2345af8cbc8/users/084a0502f701456c9c188833f0ca1239/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:38:20.953211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c106a-2651-4f44-975c-8cd1d635c397 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:20.953556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c106a-2651-4f44-975c-8cd1d635c397 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:20.954097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c106a-2651-4f44-975c-8cd1d635c397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:20.992917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 881/1786] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:20 2022] POST /identity/v3/projects => generated 365 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:20.997707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76f35c-d4e9-4d79-89c5-2534f3890945 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:21.082033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-549c106a-2651-4f44-975c-8cd1d635c397 None None] MFA Rules not processed for user `084a0502f701456c9c188833f0ca1239`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:21.083689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-549c106a-2651-4f44-975c-8cd1d635c397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:21.214802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:21.248067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76f35c-d4e9-4d79-89c5-2534f3890945 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:21.266659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360e810-c613-4dd3-a560-ca51367dc17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:21.267602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360e810-c613-4dd3-a560-ca51367dc17e admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:21.267995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360e810-c613-4dd3-a560-ca51367dc17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:21.276220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360e810-c613-4dd3-a560-ca51367dc17e admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:21.277905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3360e810-c613-4dd3-a560-ca51367dc17e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:21.281791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3360e810-c613-4dd3-a560-ca51367dc17e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:21.333854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 882/1787] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:20 2022] POST /identity/v3/users => generated 289 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:21.335403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca6177-7cae-4dc6-b6bc-e1fe0be99a06 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:21.336234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca6177-7cae-4dc6-b6bc-e1fe0be99a06 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:21.336234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca6177-7cae-4dc6-b6bc-e1fe0be99a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:21.361633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-8dca6177-7cae-4dc6-b6bc-e1fe0be99a06 None None] MFA Rules not processed for user `7d712e201f2c47a8a48b6e24559e0d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:21.363636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8dca6177-7cae-4dc6-b6bc-e1fe0be99a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:21.444542 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 883/1788] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:21.451732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b5170-c2af-42db-a023-167d773f326d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:21.451732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b5170-c2af-42db-a023-167d773f326d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:21.451732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b5170-c2af-42db-a023-167d773f326d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:21.465141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 906/1789] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:38:21.469501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c106a-2651-4f44-975c-8cd1d635c397 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:21.527984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c106a-2651-4f44-975c-8cd1d635c397 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:21.555134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309975ef-fec4-46ad-981d-7d60e1b87d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:21.556139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309975ef-fec4-46ad-981d-7d60e1b87d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:21.556613 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309975ef-fec4-46ad-981d-7d60e1b87d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:21.557487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309975ef-fec4-46ad-981d-7d60e1b87d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:21.558243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ff8b5170-c2af-42db-a023-167d773f326d None None] MFA Rules not processed for user `0aa8933ba0524fea94470931850a35e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:21.559489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ff8b5170-c2af-42db-a023-167d773f326d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:21.559767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309975ef-fec4-46ad-981d-7d60e1b87d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:21.565695 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309975ef-fec4-46ad-981d-7d60e1b87d7f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:21.579565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:21.595538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 907/1790] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:21.598823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309975ef-fec4-46ad-981d-7d60e1b87d7f admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:21.604775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-309975ef-fec4-46ad-981d-7d60e1b87d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:21.760436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 884/1791] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:21 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:21.761609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b5170-c2af-42db-a023-167d773f326d None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:21.777697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b5170-c2af-42db-a023-167d773f326d None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:21.787353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d5480-5a13-40fa-9bb4-631f43986b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:21.788354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d5480-5a13-40fa-9bb4-631f43986b24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:21.790070 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d5480-5a13-40fa-9bb4-631f43986b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:21.790425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d5480-5a13-40fa-9bb4-631f43986b24 admin admin] PATH_INFO: `/v3/projects/92d1507ce7eb43e68db979bb5b8d4357/users/5f3a9cab585645f2b870e7a4e7bb8145/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:21.829721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309975ef-fec4-46ad-981d-7d60e1b87d7f admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:21.831676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d5480-5a13-40fa-9bb4-631f43986b24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d1507ce7eb43e68db979bb5b8d4357, user_id=5f3a9cab585645f2b870e7a4e7bb8145, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:21.837557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d5480-5a13-40fa-9bb4-631f43986b24 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:21.841023 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed6b5a-f803-4c4b-9926-c9d45f3859d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:21.842002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed6b5a-f803-4c4b-9926-c9d45f3859d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:21.843005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed6b5a-f803-4c4b-9926-c9d45f3859d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:21.843345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed6b5a-f803-4c4b-9926-c9d45f3859d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:21.869432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 885/1792] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:21 2022] PUT /identity/v3/projects/92d1507ce7eb43e68db979bb5b8d4357/users/5f3a9cab585645f2b870e7a4e7bb8145/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:21.875231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d5480-5a13-40fa-9bb4-631f43986b24 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:21.876637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed6b5a-f803-4c4b-9926-c9d45f3859d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:21.878914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed6b5a-f803-4c4b-9926-c9d45f3859d0 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:21.909084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 908/1793] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:21 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:21.990218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d5480-5a13-40fa-9bb4-631f43986b24 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:22.000504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5e81f-90be-4daf-a76b-d612ce372f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:22.003586 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5e81f-90be-4daf-a76b-d612ce372f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:22.004591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5e81f-90be-4daf-a76b-d612ce372f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:22.006059 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5e81f-90be-4daf-a76b-d612ce372f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:22.007941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5e81f-90be-4daf-a76b-d612ce372f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:22.012612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5e81f-90be-4daf-a76b-d612ce372f3b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:22.055263 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 886/1794] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:22.056590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed6b5a-f803-4c4b-9926-c9d45f3859d0 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:22.066495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5e81f-90be-4daf-a76b-d612ce372f3b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:22.074124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed6b5a-f803-4c4b-9926-c9d45f3859d0 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:22.139187 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5e81f-90be-4daf-a76b-d612ce372f3b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:22.161601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425f07e-e532-4342-9929-0848e2531f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:22.161601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425f07e-e532-4342-9929-0848e2531f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:22.161601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425f07e-e532-4342-9929-0848e2531f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:22.162300 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425f07e-e532-4342-9929-0848e2531f1b admin admin] PATH_INFO: `/v3/projects/92d1507ce7eb43e68db979bb5b8d4357/users/5f3a9cab585645f2b870e7a4e7bb8145/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:22.171100 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425f07e-e532-4342-9929-0848e2531f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d1507ce7eb43e68db979bb5b8d4357, user_id=5f3a9cab585645f2b870e7a4e7bb8145, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:22.182084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425f07e-e532-4342-9929-0848e2531f1b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:22.248819 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 887/1795] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:22 2022] PUT /identity/v3/projects/92d1507ce7eb43e68db979bb5b8d4357/users/5f3a9cab585645f2b870e7a4e7bb8145/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:22.253861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3654ce-d596-45cb-bb83-10078fba872b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:22.254920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3654ce-d596-45cb-bb83-10078fba872b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:22.257802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3654ce-d596-45cb-bb83-10078fba872b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:22.315941 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766706e-b5bd-40ec-b20e-72d67aa7108c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:22.319480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766706e-b5bd-40ec-b20e-72d67aa7108c service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:22.319638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766706e-b5bd-40ec-b20e-72d67aa7108c service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:22.322718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766706e-b5bd-40ec-b20e-72d67aa7108c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:22.432658 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-2f3654ce-d596-45cb-bb83-10078fba872b None None] MFA Rules not processed for user `5f3a9cab585645f2b870e7a4e7bb8145`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:22.437907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2f3654ce-d596-45cb-bb83-10078fba872b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:22.594434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:22.631786 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766706e-b5bd-40ec-b20e-72d67aa7108c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:22.634158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766706e-b5bd-40ec-b20e-72d67aa7108c service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:22.649613 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 909/1796] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:22.662142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 888/1797] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:22.905951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766706e-b5bd-40ec-b20e-72d67aa7108c service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:22.907815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3766706e-b5bd-40ec-b20e-72d67aa7108c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:22.931765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766706e-b5bd-40ec-b20e-72d67aa7108c service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:22.941783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:23.133084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d4d20-ba63-4304-9013-278a040816a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:23.134080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d4d20-ba63-4304-9013-278a040816a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:23.134369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d4d20-ba63-4304-9013-278a040816a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:23.134640 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d4d20-ba63-4304-9013-278a040816a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:23.165272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d4d20-ba63-4304-9013-278a040816a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:23.166928 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d4d20-ba63-4304-9013-278a040816a6 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:23.193147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 910/1798] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:23.665309 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3654ce-d596-45cb-bb83-10078fba872b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:23.685117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3654ce-d596-45cb-bb83-10078fba872b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:23.699832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa51a88-8515-4b3a-b6ee-1378c319af1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:23.700402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa51a88-8515-4b3a-b6ee-1378c319af1a service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:23.700687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa51a88-8515-4b3a-b6ee-1378c319af1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:23.701023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa51a88-8515-4b3a-b6ee-1378c319af1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:23.778057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4c525-e678-4859-b4f4-aa634189c8c8 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:23.780227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4c525-e678-4859-b4f4-aa634189c8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:23.781198 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4c525-e678-4859-b4f4-aa634189c8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:23.942714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-4aa4c525-e678-4859-b4f4-aa634189c8c8 None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:23.945588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4aa4c525-e678-4859-b4f4-aa634189c8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:23.981739 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa51a88-8515-4b3a-b6ee-1378c319af1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:23.989483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa51a88-8515-4b3a-b6ee-1378c319af1a service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:24.012119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 889/1799] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:24.015128 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156db5-dfd4-4ce9-ae08-f79b4f8b5f1b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:24.017919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156db5-dfd4-4ce9-ae08-f79b4f8b5f1b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:24.017919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156db5-dfd4-4ce9-ae08-f79b4f8b5f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:24.066054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-24156db5-dfd4-4ce9-ae08-f79b4f8b5f1b None None] MFA Rules not processed for user `fe0fcdeafae645a582afcee3dde79ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:24.069835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-24156db5-dfd4-4ce9-ae08-f79b4f8b5f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:24.104529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 911/1800] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:38:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:24.231762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 890/1801] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:24.419220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4c525-e678-4859-b4f4-aa634189c8c8 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:24.420745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4aa4c525-e678-4859-b4f4-aa634189c8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:24.651711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:24.668330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4c525-e678-4859-b4f4-aa634189c8c8 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:24.682152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2b427-98d6-4103-84b5-196fde131d07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:24.682823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2b427-98d6-4103-84b5-196fde131d07 service swift] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:24.683092 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2b427-98d6-4103-84b5-196fde131d07 service swift] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:24.683377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2b427-98d6-4103-84b5-196fde131d07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:24.702009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2b427-98d6-4103-84b5-196fde131d07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:24.704028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2b427-98d6-4103-84b5-196fde131d07 service swift] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:24.735716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 912/1802] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:38:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 08:38:26.811964 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2839a7c-d2b6-498c-b722-f11525e652bf None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:26.813178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2839a7c-d2b6-498c-b722-f11525e652bf None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:26.813699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2839a7c-d2b6-498c-b722-f11525e652bf None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:26.816794 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 891/1803] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:38:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:38:26.896229 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad407bfa-1f6e-4316-8469-91631a13ef0a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:26.897820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad407bfa-1f6e-4316-8469-91631a13ef0a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:26.898652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad407bfa-1f6e-4316-8469-91631a13ef0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:26.991351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-ad407bfa-1f6e-4316-8469-91631a13ef0a None None] MFA Rules not processed for user `6b40ee263df74e58bf3f958c01eb742f`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:27.006756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ad407bfa-1f6e-4316-8469-91631a13ef0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:27.340225 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 913/1804] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:38:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:27.490192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2839a7c-d2b6-498c-b722-f11525e652bf None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:27.515170 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2839a7c-d2b6-498c-b722-f11525e652bf None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:27.525278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fc6c8-ca15-40e7-83e0-0f3cbece5588 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:27.526396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fc6c8-ca15-40e7-83e0-0f3cbece5588 service placement] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:27.526844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fc6c8-ca15-40e7-83e0-0f3cbece5588 service placement] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:27.527357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fc6c8-ca15-40e7-83e0-0f3cbece5588 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:27.545562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fc6c8-ca15-40e7-83e0-0f3cbece5588 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:27.552495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fc6c8-ca15-40e7-83e0-0f3cbece5588 service placement] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:27.605431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 892/1805] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:27.871864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0a69f-2fe3-4003-b9dc-c3bfa36b9266 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:27.872463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0a69f-2fe3-4003-b9dc-c3bfa36b9266 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:27.873625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0a69f-2fe3-4003-b9dc-c3bfa36b9266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:27.929747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-14c0a69f-2fe3-4003-b9dc-c3bfa36b9266 None None] MFA Rules not processed for user `7e3b9ceb95374f1dbf356122c8ed463c`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:27.933999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-14c0a69f-2fe3-4003-b9dc-c3bfa36b9266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:28.221427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 914/1806] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:29.471616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd40292-821a-4647-bc15-b192f8c432f9 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:29.472637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd40292-821a-4647-bc15-b192f8c432f9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:29.473153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd40292-821a-4647-bc15-b192f8c432f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:29.524293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-bbd40292-821a-4647-bc15-b192f8c432f9 None None] MFA Rules not processed for user `736efafe9fc94b45bc75566b1696c498`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:29.525914 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-bbd40292-821a-4647-bc15-b192f8c432f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:29.722368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 893/1807] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Sep 28 08:38:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:29.868853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0a69f-2fe3-4003-b9dc-c3bfa36b9266 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:29.870790 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-14c0a69f-2fe3-4003-b9dc-c3bfa36b9266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:29.899413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0a69f-2fe3-4003-b9dc-c3bfa36b9266 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:29.911595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aacab8-918e-4c5f-b638-87f6f65001ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:29.912680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aacab8-918e-4c5f-b638-87f6f65001ec service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:29.913262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aacab8-918e-4c5f-b638-87f6f65001ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:29.914138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aacab8-918e-4c5f-b638-87f6f65001ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:29.942024 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aacab8-918e-4c5f-b638-87f6f65001ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:29.944492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aacab8-918e-4c5f-b638-87f6f65001ec service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:29.971774 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 915/1808] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:38:30.549405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd40292-821a-4647-bc15-b192f8c432f9 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:30.634975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:30.741942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd40292-821a-4647-bc15-b192f8c432f9 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:30.762693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4764a0e-1318-43e0-af49-7e8033c45dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:30.768301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4764a0e-1318-43e0-af49-7e8033c45dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:30.770822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4764a0e-1318-43e0-af49-7e8033c45dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:30.771352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4764a0e-1318-43e0-af49-7e8033c45dac admin admin] PATH_INFO: `/v3/users/832fec615e5149e4bcaa807b7a1813df` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:30.775911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4764a0e-1318-43e0-af49-7e8033c45dac admin admin] RBAC: Authorizing `identity:delete_user(user_id=832fec615e5149e4bcaa807b7a1813df)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:30.789998 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4764a0e-1318-43e0-af49-7e8033c45dac admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:30.804556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aacab8-918e-4c5f-b638-87f6f65001ec service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:30.822495 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aacab8-918e-4c5f-b638-87f6f65001ec service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:30.843101 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11359a-9afd-4468-977d-41fab69b68c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:30.843445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11359a-9afd-4468-977d-41fab69b68c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:30.848679 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11359a-9afd-4468-977d-41fab69b68c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:30.849057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11359a-9afd-4468-977d-41fab69b68c7 admin admin] PATH_INFO: `/v3/users/bf69342d99824c90bce94e7e47ebd48a` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:30.860466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11359a-9afd-4468-977d-41fab69b68c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf69342d99824c90bce94e7e47ebd48a)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:30.873730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11359a-9afd-4468-977d-41fab69b68c7 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:30.985985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d4764a0e-1318-43e0-af49-7e8033c45dac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '832fec615e5149e4bcaa807b7a1813df'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:31.009471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d4764a0e-1318-43e0-af49-7e8033c45dac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '832fec615e5149e4bcaa807b7a1813df'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:31.012156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d4764a0e-1318-43e0-af49-7e8033c45dac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '832fec615e5149e4bcaa807b7a1813df'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:31.014738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d4764a0e-1318-43e0-af49-7e8033c45dac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '832fec615e5149e4bcaa807b7a1813df'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:31.037987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d4764a0e-1318-43e0-af49-7e8033c45dac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '832fec615e5149e4bcaa807b7a1813df'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:31.063678 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bc11359a-9afd-4468-977d-41fab69b68c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf69342d99824c90bce94e7e47ebd48a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:31.106011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 894/1809] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:30 2022] DELETE /identity/v3/users/832fec615e5149e4bcaa807b7a1813df => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:31.111733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc288471-0ef5-4280-ba64-158376657857 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:31.112152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc288471-0ef5-4280-ba64-158376657857 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:31.112764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc288471-0ef5-4280-ba64-158376657857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:31.117465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bc11359a-9afd-4468-977d-41fab69b68c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf69342d99824c90bce94e7e47ebd48a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:31.118570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bc11359a-9afd-4468-977d-41fab69b68c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf69342d99824c90bce94e7e47ebd48a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:31.137305 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bc11359a-9afd-4468-977d-41fab69b68c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf69342d99824c90bce94e7e47ebd48a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:31.139916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bc11359a-9afd-4468-977d-41fab69b68c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf69342d99824c90bce94e7e47ebd48a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:31.142427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:31.171306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 916/1810] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:30 2022] DELETE /identity/v3/users/bf69342d99824c90bce94e7e47ebd48a => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:31.185660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e173f-6fbb-43b6-a196-b375b2e137fa None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:31.185660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e173f-6fbb-43b6-a196-b375b2e137fa None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:31.185660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e173f-6fbb-43b6-a196-b375b2e137fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:31.200070 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-bc288471-0ef5-4280-ba64-158376657857 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:31.202375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-bc288471-0ef5-4280-ba64-158376657857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:31.230855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-359e173f-6fbb-43b6-a196-b375b2e137fa None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:31.232508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-359e173f-6fbb-43b6-a196-b375b2e137fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:31.445026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 917/1811] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:38:31.445769 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 895/1812] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:31.579284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e173f-6fbb-43b6-a196-b375b2e137fa None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:31.580807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-359e173f-6fbb-43b6-a196-b375b2e137fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:31.769481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e173f-6fbb-43b6-a196-b375b2e137fa None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:31.790013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c4835-ab2d-449d-affc-40779b76a7dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:31.792380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c4835-ab2d-449d-affc-40779b76a7dc service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:31.792789 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c4835-ab2d-449d-affc-40779b76a7dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:31.794336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c4835-ab2d-449d-affc-40779b76a7dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:31.824573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c4835-ab2d-449d-affc-40779b76a7dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:31.826194 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c4835-ab2d-449d-affc-40779b76a7dc service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:31.846037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 918/1813] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:38:31.970665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc288471-0ef5-4280-ba64-158376657857 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:31.983583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc288471-0ef5-4280-ba64-158376657857 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:32.014231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec54364-45ef-411a-87b7-106e27a558c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:32.015116 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec54364-45ef-411a-87b7-106e27a558c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:32.016858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec54364-45ef-411a-87b7-106e27a558c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:32.017703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec54364-45ef-411a-87b7-106e27a558c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:32.061658 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec54364-45ef-411a-87b7-106e27a558c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:32.063100 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec54364-45ef-411a-87b7-106e27a558c2 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:32.170958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 896/1814] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:32.564170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c4835-ab2d-449d-affc-40779b76a7dc service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:32.566555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-985c4835-ab2d-449d-affc-40779b76a7dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:32.626399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c4835-ab2d-449d-affc-40779b76a7dc service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:32.651251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:32.652735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:32.653343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:32.654459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] PATH_INFO: `/v3/users/b65d9e12f6584b228134af177d536780` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:32.658923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65d9e12f6584b228134af177d536780)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:32.666339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:32.900524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65d9e12f6584b228134af177d536780'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:32.983232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65d9e12f6584b228134af177d536780'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:32.985376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65d9e12f6584b228134af177d536780'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:33.020612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65d9e12f6584b228134af177d536780'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:33.021924 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65d9e12f6584b228134af177d536780'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:33.071051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 919/1815] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:32 2022] DELETE /identity/v3/users/b65d9e12f6584b228134af177d536780 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:34.540135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec54364-45ef-411a-87b7-106e27a558c2 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:34.542739 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-0ec54364-45ef-411a-87b7-106e27a558c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:34.608781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec54364-45ef-411a-87b7-106e27a558c2 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:34.786210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:34.907525 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:34.908198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:34.908472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:34.908739 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] PATH_INFO: `/v3/projects/2f615366bc9941e9abc511a35f389344` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:34.910582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f615366bc9941e9abc511a35f389344)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:34.912536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:34.954758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:34.956286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:34.991683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e224f6-35ad-4505-96b9-fd613f8a1583 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:35.018072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:35.023992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:35.024281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:35.026266 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] PATH_INFO: `/v3/projects/52fe6054bbd34fb5b302a30c7b9e6c3d` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:35.038196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52fe6054bbd34fb5b302a30c7b9e6c3d)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:35.044813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:35.050663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:35.156649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f615366bc9941e9abc511a35f389344', 'request_id': 'req-75f92c41-be52-43f8-b4ab-477633ed82d1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.179972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f615366bc9941e9abc511a35f389344', 'request_id': 'req-75f92c41-be52-43f8-b4ab-477633ed82d1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.203407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f615366bc9941e9abc511a35f389344', 'request_id': 'req-75f92c41-be52-43f8-b4ab-477633ed82d1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.247562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] The token cache is being invalidate because project 2f615366bc9941e9abc511a35f389344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:35.247867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.266427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52fe6054bbd34fb5b302a30c7b9e6c3d', 'request_id': 'req-190e57f9-0696-4a06-8d76-d3ecd69f2123'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.270412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.276863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52fe6054bbd34fb5b302a30c7b9e6c3d', 'request_id': 'req-190e57f9-0696-4a06-8d76-d3ecd69f2123'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.280575 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 897/1816] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:34 2022] DELETE /identity/v3/projects/2f615366bc9941e9abc511a35f389344 => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:35.283324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:35.287013 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:35.309827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52fe6054bbd34fb5b302a30c7b9e6c3d', 'request_id': 'req-190e57f9-0696-4a06-8d76-d3ecd69f2123'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.336769 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] The token cache is being invalidate because project 52fe6054bbd34fb5b302a30c7b9e6c3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:35.337405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.338389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.340279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 920/1817] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:34 2022] DELETE /identity/v3/projects/52fe6054bbd34fb5b302a30c7b9e6c3d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:38:35.345857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:35.349720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:35.588380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e57f9-0696-4a06-8d76-d3ecd69f2123 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:35.613246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60bde9-e25d-46ad-9f7d-07c87f613e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:35.614378 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60bde9-e25d-46ad-9f7d-07c87f613e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:35.615861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60bde9-e25d-46ad-9f7d-07c87f613e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:35.617362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60bde9-e25d-46ad-9f7d-07c87f613e63 admin admin] PATH_INFO: `/v3/users/084a0502f701456c9c188833f0ca1239` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:35.619487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60bde9-e25d-46ad-9f7d-07c87f613e63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=084a0502f701456c9c188833f0ca1239)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:35.626675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60bde9-e25d-46ad-9f7d-07c87f613e63 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:35.643503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f92c41-be52-43f8-b4ab-477633ed82d1 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:35.660040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4cc68-d30a-42ac-8437-3caff1ed2f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:35.661904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4cc68-d30a-42ac-8437-3caff1ed2f1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:35.664421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4cc68-d30a-42ac-8437-3caff1ed2f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:35.665039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4cc68-d30a-42ac-8437-3caff1ed2f1e admin admin] PATH_INFO: `/v3/users/5f3a9cab585645f2b870e7a4e7bb8145` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:35.671858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4cc68-d30a-42ac-8437-3caff1ed2f1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f3a9cab585645f2b870e7a4e7bb8145)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:35.674343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4cc68-d30a-42ac-8437-3caff1ed2f1e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:35.783145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-de60bde9-e25d-46ad-9f7d-07c87f613e63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '084a0502f701456c9c188833f0ca1239'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.859410 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8cf4cc68-d30a-42ac-8437-3caff1ed2f1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f3a9cab585645f2b870e7a4e7bb8145'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.860053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-de60bde9-e25d-46ad-9f7d-07c87f613e63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '084a0502f701456c9c188833f0ca1239'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.865460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-de60bde9-e25d-46ad-9f7d-07c87f613e63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '084a0502f701456c9c188833f0ca1239'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.891754 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8cf4cc68-d30a-42ac-8437-3caff1ed2f1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f3a9cab585645f2b870e7a4e7bb8145'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.893398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-de60bde9-e25d-46ad-9f7d-07c87f613e63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '084a0502f701456c9c188833f0ca1239'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.894602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8cf4cc68-d30a-42ac-8437-3caff1ed2f1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f3a9cab585645f2b870e7a4e7bb8145'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.895624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-de60bde9-e25d-46ad-9f7d-07c87f613e63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '084a0502f701456c9c188833f0ca1239'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.898378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8cf4cc68-d30a-42ac-8437-3caff1ed2f1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f3a9cab585645f2b870e7a4e7bb8145'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.920310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8cf4cc68-d30a-42ac-8437-3caff1ed2f1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f3a9cab585645f2b870e7a4e7bb8145'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:35.931461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:35.945602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 921/1818] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:35 2022] DELETE /identity/v3/users/084a0502f701456c9c188833f0ca1239 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:35.988474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 898/1819] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:35 2022] DELETE /identity/v3/users/5f3a9cab585645f2b870e7a4e7bb8145 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:36.125404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47deba14-e7a4-40cd-890d-6686ec920166 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:36.125969 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47deba14-e7a4-40cd-890d-6686ec920166 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:36.125969 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47deba14-e7a4-40cd-890d-6686ec920166 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:36.127320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 922/1820] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:38:36.147254 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36e537-4475-4d5d-8b41-c4fcd64ac838 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:36.147692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36e537-4475-4d5d-8b41-c4fcd64ac838 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:36.148079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36e537-4475-4d5d-8b41-c4fcd64ac838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:36.239310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-fa36e537-4475-4d5d-8b41-c4fcd64ac838 None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:36.243003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-fa36e537-4475-4d5d-8b41-c4fcd64ac838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:36.358197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47deba14-e7a4-40cd-890d-6686ec920166 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:36.362311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-47deba14-e7a4-40cd-890d-6686ec920166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:36.406673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:36.623596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47deba14-e7a4-40cd-890d-6686ec920166 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:36.649797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 899/1821] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:38:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 504 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:36.654562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:36.655290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:36.655994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:36.656249 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] PATH_INFO: `/v3/projects/94f139132b1c489d8006b88c91b754da` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:36.657806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f139132b1c489d8006b88c91b754da)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:36.659787 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:36.787354 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f139132b1c489d8006b88c91b754da', 'request_id': 'req-09e62944-3730-4542-97c5-af12583766bc'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:36.801193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f139132b1c489d8006b88c91b754da', 'request_id': 'req-09e62944-3730-4542-97c5-af12583766bc'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:36.817494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f139132b1c489d8006b88c91b754da', 'request_id': 'req-09e62944-3730-4542-97c5-af12583766bc'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:36.864952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] The token cache is being invalidate because project 94f139132b1c489d8006b88c91b754da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:36.865680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:36.866639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:36.869308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 923/1822] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:36 2022] DELETE /identity/v3/projects/94f139132b1c489d8006b88c91b754da => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:38:36.875322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36e537-4475-4d5d-8b41-c4fcd64ac838 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:36.895958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36e537-4475-4d5d-8b41-c4fcd64ac838 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:37.103245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:37.109266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:37.110067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:37.110579 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] PATH_INFO: `/v3/users/0aa8933ba0524fea94470931850a35e6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:37.112155 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa8933ba0524fea94470931850a35e6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:37.117975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:37.251549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa8933ba0524fea94470931850a35e6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:37.274875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa8933ba0524fea94470931850a35e6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:37.275876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa8933ba0524fea94470931850a35e6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:37.276775 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa8933ba0524fea94470931850a35e6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:37.288157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa8933ba0524fea94470931850a35e6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:37.303166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:37.326890 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:37.331167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:37.433767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 900/1823] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:36 2022] DELETE /identity/v3/users/0aa8933ba0524fea94470931850a35e6 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:37.646462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e62944-3730-4542-97c5-af12583766bc admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:37.762036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:37.906490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360637a3-004c-4f8c-909d-508cd5ff76c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:37.907136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360637a3-004c-4f8c-909d-508cd5ff76c5 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:37.907418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360637a3-004c-4f8c-909d-508cd5ff76c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:37.907769 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360637a3-004c-4f8c-909d-508cd5ff76c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:37.920082 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-360637a3-004c-4f8c-909d-508cd5ff76c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:37.959084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360637a3-004c-4f8c-909d-508cd5ff76c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:37.960955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360637a3-004c-4f8c-909d-508cd5ff76c5 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:37.989155 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 924/1824] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:38.914697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:38.916811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:39.043639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da5ca8-38d1-4b48-9d7b-0d397d617739 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:39.057624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:39.059093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:39.059093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:39.059093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] PATH_INFO: `/v3/projects/9e2d753c4cff4ded80d6c2345af8cbc8` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:39.060135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e2d753c4cff4ded80d6c2345af8cbc8)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:39.062158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:39.169766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e2d753c4cff4ded80d6c2345af8cbc8', 'request_id': 'req-d0103966-2f17-4263-820c-f2a86d0c6682'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.181507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e2d753c4cff4ded80d6c2345af8cbc8', 'request_id': 'req-d0103966-2f17-4263-820c-f2a86d0c6682'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.220343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360637a3-004c-4f8c-909d-508cd5ff76c5 service nova] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:39.222200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-360637a3-004c-4f8c-909d-508cd5ff76c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:39.225796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e2d753c4cff4ded80d6c2345af8cbc8', 'request_id': 'req-d0103966-2f17-4263-820c-f2a86d0c6682'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.245937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360637a3-004c-4f8c-909d-508cd5ff76c5 service nova] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:39.275438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] The token cache is being invalidate because project 9e2d753c4cff4ded80d6c2345af8cbc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:39.275796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.283823 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.285714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 901/1825] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:38 2022] DELETE /identity/v3/projects/9e2d753c4cff4ded80d6c2345af8cbc8 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:39.292461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:39.293924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:39.344519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24df8d-d1ca-4e15-a7be-d32e01f56e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:39.345547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24df8d-d1ca-4e15-a7be-d32e01f56e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:39.346059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24df8d-d1ca-4e15-a7be-d32e01f56e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:39.346492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24df8d-d1ca-4e15-a7be-d32e01f56e50 admin admin] PATH_INFO: `/v3/projects/1cf0587f5b0146738736237e64f33d66` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:39.347883 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24df8d-d1ca-4e15-a7be-d32e01f56e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf0587f5b0146738736237e64f33d66)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:39.350311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24df8d-d1ca-4e15-a7be-d32e01f56e50 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:39.414413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-df24df8d-d1ca-4e15-a7be-d32e01f56e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf0587f5b0146738736237e64f33d66', 'request_id': 'req-df24df8d-d1ca-4e15-a7be-d32e01f56e50'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.430536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0103966-2f17-4263-820c-f2a86d0c6682 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:39.437003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-df24df8d-d1ca-4e15-a7be-d32e01f56e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf0587f5b0146738736237e64f33d66', 'request_id': 'req-df24df8d-d1ca-4e15-a7be-d32e01f56e50'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.447641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-df24df8d-d1ca-4e15-a7be-d32e01f56e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf0587f5b0146738736237e64f33d66', 'request_id': 'req-df24df8d-d1ca-4e15-a7be-d32e01f56e50'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.484208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:39.487717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-df24df8d-d1ca-4e15-a7be-d32e01f56e50 admin admin] The token cache is being invalidate because project 1cf0587f5b0146738736237e64f33d66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:39.488017 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-df24df8d-d1ca-4e15-a7be-d32e01f56e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.489218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-df24df8d-d1ca-4e15-a7be-d32e01f56e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.499432 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 925/1826] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:39 2022] DELETE /identity/v3/projects/1cf0587f5b0146738736237e64f33d66 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:39.504413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858da604-27da-40b0-ac89-d54637da23d8 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:39.504719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858da604-27da-40b0-ac89-d54637da23d8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:39.504955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858da604-27da-40b0-ac89-d54637da23d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:39.529671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc0267-0cd7-472d-bc53-51e481d276f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:39.530393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc0267-0cd7-472d-bc53-51e481d276f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:39.530727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc0267-0cd7-472d-bc53-51e481d276f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:39.531008 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc0267-0cd7-472d-bc53-51e481d276f1 admin admin] PATH_INFO: `/v3/projects/92d1507ce7eb43e68db979bb5b8d4357` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:39.532915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc0267-0cd7-472d-bc53-51e481d276f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92d1507ce7eb43e68db979bb5b8d4357)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:39.535164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc0267-0cd7-472d-bc53-51e481d276f1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:39.568475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-858da604-27da-40b0-ac89-d54637da23d8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:39.570904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-858da604-27da-40b0-ac89-d54637da23d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:39.585923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 926/1827] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:39.587918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6496979-c74c-4b52-8ac5-f77816a04a47 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:39.588893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6496979-c74c-4b52-8ac5-f77816a04a47 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:39.589870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6496979-c74c-4b52-8ac5-f77816a04a47 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:39.597581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 927/1828] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:38:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:38:39.601412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f772c9-f12e-48d9-b80a-ad17f6c7cba5 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:39.603441 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f772c9-f12e-48d9-b80a-ad17f6c7cba5 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:39.607339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f772c9-f12e-48d9-b80a-ad17f6c7cba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:39.611441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7fdc0267-0cd7-472d-bc53-51e481d276f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92d1507ce7eb43e68db979bb5b8d4357', 'request_id': 'req-7fdc0267-0cd7-472d-bc53-51e481d276f1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.624114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7fdc0267-0cd7-472d-bc53-51e481d276f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92d1507ce7eb43e68db979bb5b8d4357', 'request_id': 'req-7fdc0267-0cd7-472d-bc53-51e481d276f1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.643898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7fdc0267-0cd7-472d-bc53-51e481d276f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92d1507ce7eb43e68db979bb5b8d4357', 'request_id': 'req-7fdc0267-0cd7-472d-bc53-51e481d276f1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.668358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b5f772c9-f12e-48d9-b80a-ad17f6c7cba5 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:39.670028 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7fdc0267-0cd7-472d-bc53-51e481d276f1 admin admin] The token cache is being invalidate because project 92d1507ce7eb43e68db979bb5b8d4357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:39.670334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7fdc0267-0cd7-472d-bc53-51e481d276f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.670714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b5f772c9-f12e-48d9-b80a-ad17f6c7cba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:39.671018 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7fdc0267-0cd7-472d-bc53-51e481d276f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:39.672782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 902/1829] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:39 2022] DELETE /identity/v3/projects/92d1507ce7eb43e68db979bb5b8d4357 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:39.679272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b5332-efa3-4cba-8ed2-11081cb11775 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:39.679570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b5332-efa3-4cba-8ed2-11081cb11775 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:39.679852 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b5332-efa3-4cba-8ed2-11081cb11775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:39.736946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-8b1b5332-efa3-4cba-8ed2-11081cb11775 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:39.738015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8b1b5332-efa3-4cba-8ed2-11081cb11775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:39.745508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:39.777740 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 928/1830] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:39.777740 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffdf04-c761-4eaf-94ad-d62287e11197 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:39.777740 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffdf04-c761-4eaf-94ad-d62287e11197 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:39.777740 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffdf04-c761-4eaf-94ad-d62287e11197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:39.813030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-84ffdf04-c761-4eaf-94ad-d62287e11197 None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:39.814451 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-84ffdf04-c761-4eaf-94ad-d62287e11197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:39.875691 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 903/1831] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:39.878436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331cb8e-d73d-4889-9766-352a4854b4b2 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:39.878436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331cb8e-d73d-4889-9766-352a4854b4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:39.878883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331cb8e-d73d-4889-9766-352a4854b4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:39.937027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-f331cb8e-d73d-4889-9766-352a4854b4b2 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:39.943520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f331cb8e-d73d-4889-9766-352a4854b4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:39.955236 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 929/1832] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:38:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:39.956131 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 904/1833] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:39.957318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331cb8e-d73d-4889-9766-352a4854b4b2 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:39.961763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a05e6-377b-4f0d-b754-feedce398ce6 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:39.962057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a05e6-377b-4f0d-b754-feedce398ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:39.962426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870a05e6-377b-4f0d-b754-feedce398ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:39.969553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331cb8e-d73d-4889-9766-352a4854b4b2 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:39.977854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a5ffa-24a2-4e93-bdb4-c3d1e7e9620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:39.978476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a5ffa-24a2-4e93-bdb4-c3d1e7e9620a admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:39.978802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a5ffa-24a2-4e93-bdb4-c3d1e7e9620a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:39.979022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a5ffa-24a2-4e93-bdb4-c3d1e7e9620a admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:39.980290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a5ffa-24a2-4e93-bdb4-c3d1e7e9620a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:38:39.980290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a5ffa-24a2-4e93-bdb4-c3d1e7e9620a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:39.981853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a5ffa-24a2-4e93-bdb4-c3d1e7e9620a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:39.995379 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-870a05e6-377b-4f0d-b754-feedce398ce6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:39.996562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-870a05e6-377b-4f0d-b754-feedce398ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:39.998268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 905/1834] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:38:39.999780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ee568-dea3-4261-be26-06b01d084081 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.000074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ee568-dea3-4261-be26-06b01d084081 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.000583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ee568-dea3-4261-be26-06b01d084081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.005485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 930/1835] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:40.009887 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a05e6-377b-4f0d-b754-feedce398ce6 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.011294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-870a05e6-377b-4f0d-b754-feedce398ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:40.024630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-401ee568-dea3-4261-be26-06b01d084081 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:40.025908 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-401ee568-dea3-4261-be26-06b01d084081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:40.033141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870a05e6-377b-4f0d-b754-feedce398ce6 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.037158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 906/1836] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:40.037989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ee568-dea3-4261-be26-06b01d084081 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.046461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91da22f-6e32-4bbd-9ac5-eca11f41338d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.047481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91da22f-6e32-4bbd-9ac5-eca11f41338d service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.047970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91da22f-6e32-4bbd-9ac5-eca11f41338d service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.048472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91da22f-6e32-4bbd-9ac5-eca11f41338d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.054405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ee568-dea3-4261-be26-06b01d084081 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.063125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c5361-06cb-4c28-aa13-2dd94ed9693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.063338 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91da22f-6e32-4bbd-9ac5-eca11f41338d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.064231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c5361-06cb-4c28-aa13-2dd94ed9693f admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.064798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c5361-06cb-4c28-aa13-2dd94ed9693f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.065080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c5361-06cb-4c28-aa13-2dd94ed9693f admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.065581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91da22f-6e32-4bbd-9ac5-eca11f41338d service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.066052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c5361-06cb-4c28-aa13-2dd94ed9693f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.069870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c5361-06cb-4c28-aa13-2dd94ed9693f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.079708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 931/1837] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:40.086403 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91da22f-6e32-4bbd-9ac5-eca11f41338d service nova] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.101451 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91da22f-6e32-4bbd-9ac5-eca11f41338d service nova] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.107622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 907/1838] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:40 2022] POST /identity/v3/projects => generated 369 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:40.108630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c5361-06cb-4c28-aa13-2dd94ed9693f admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.113050 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5529c1-e547-45d7-aac8-cbf3ee53e6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.113703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5529c1-e547-45d7-aac8-cbf3ee53e6ce admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.113982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5529c1-e547-45d7-aac8-cbf3ee53e6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.114252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5529c1-e547-45d7-aac8-cbf3ee53e6ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.114997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5529c1-e547-45d7-aac8-cbf3ee53e6ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:38:40.115300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5529c1-e547-45d7-aac8-cbf3ee53e6ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.117389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5529c1-e547-45d7-aac8-cbf3ee53e6ce admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.133072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 932/1839] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:38:40.134187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5529c1-e547-45d7-aac8-cbf3ee53e6ce admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.138607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:40.145285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5529c1-e547-45d7-aac8-cbf3ee53e6ce admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.212460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c5361-06cb-4c28-aa13-2dd94ed9693f admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.220827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fe99d-f8f7-4fda-83af-ac4ad72c7392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.221574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fe99d-f8f7-4fda-83af-ac4ad72c7392 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.221912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fe99d-f8f7-4fda-83af-ac4ad72c7392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.222040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fe99d-f8f7-4fda-83af-ac4ad72c7392 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.223082 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fe99d-f8f7-4fda-83af-ac4ad72c7392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:38:40.223082 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fe99d-f8f7-4fda-83af-ac4ad72c7392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.223976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caddb0-9e7f-413d-8110-a62363a61a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.225187 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fe99d-f8f7-4fda-83af-ac4ad72c7392 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.225517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caddb0-9e7f-413d-8110-a62363a61a98 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.225517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caddb0-9e7f-413d-8110-a62363a61a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.225517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caddb0-9e7f-413d-8110-a62363a61a98 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.226288 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49caddb0-9e7f-413d-8110-a62363a61a98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.228420 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49caddb0-9e7f-413d-8110-a62363a61a98 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.240981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 908/1840] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:38:40.242000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fe99d-f8f7-4fda-83af-ac4ad72c7392 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.253854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fe99d-f8f7-4fda-83af-ac4ad72c7392 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.255792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 933/1841] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:40 2022] POST /identity/v3/users => generated 285 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:40.259949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caddb0-9e7f-413d-8110-a62363a61a98 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.261531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf217c-f4ef-438a-ae4d-167bf1341589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.262358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf217c-f4ef-438a-ae4d-167bf1341589 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.262800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf217c-f4ef-438a-ae4d-167bf1341589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.263901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf217c-f4ef-438a-ae4d-167bf1341589 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.264863 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf217c-f4ef-438a-ae4d-167bf1341589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.267641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf217c-f4ef-438a-ae4d-167bf1341589 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.272429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caddb0-9e7f-413d-8110-a62363a61a98 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.278419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35c136-eb86-4cfc-b59a-c2dffcb12169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.279627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35c136-eb86-4cfc-b59a-c2dffcb12169 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.280069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35c136-eb86-4cfc-b59a-c2dffcb12169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.280349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35c136-eb86-4cfc-b59a-c2dffcb12169 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.281000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35c136-eb86-4cfc-b59a-c2dffcb12169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.283143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35c136-eb86-4cfc-b59a-c2dffcb12169 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.320886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 934/1842] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:40 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:40.322086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35c136-eb86-4cfc-b59a-c2dffcb12169 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.343104 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:40.343726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 909/1843] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:40 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:40.344907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf217c-f4ef-438a-ae4d-167bf1341589 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.397706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35c136-eb86-4cfc-b59a-c2dffcb12169 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.420886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf217c-f4ef-438a-ae4d-167bf1341589 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.425915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07276a03-0bdf-4ed6-a4a4-ea8e361e5f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.426849 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07276a03-0bdf-4ed6-a4a4-ea8e361e5f12 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.427143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07276a03-0bdf-4ed6-a4a4-ea8e361e5f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.427418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07276a03-0bdf-4ed6-a4a4-ea8e361e5f12 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.428101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07276a03-0bdf-4ed6-a4a4-ea8e361e5f12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.430408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07276a03-0bdf-4ed6-a4a4-ea8e361e5f12 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.458980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 910/1844] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:40 2022] POST /identity/v3/users => generated 284 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:40.460318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07276a03-0bdf-4ed6-a4a4-ea8e361e5f12 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.477592 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07276a03-0bdf-4ed6-a4a4-ea8e361e5f12 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.479557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195797f3-599f-4770-8ef2-afbc01ce32f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.480298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195797f3-599f-4770-8ef2-afbc01ce32f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.480595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195797f3-599f-4770-8ef2-afbc01ce32f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.480873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195797f3-599f-4770-8ef2-afbc01ce32f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.481598 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195797f3-599f-4770-8ef2-afbc01ce32f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.482930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c91b6-3722-4267-927c-6a8d5f18af8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.483479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195797f3-599f-4770-8ef2-afbc01ce32f0 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.483672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c91b6-3722-4267-927c-6a8d5f18af8e admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.483965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c91b6-3722-4267-927c-6a8d5f18af8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.484267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c91b6-3722-4267-927c-6a8d5f18af8e admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.484903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c91b6-3722-4267-927c-6a8d5f18af8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.486837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c91b6-3722-4267-927c-6a8d5f18af8e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.494212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 935/1845] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:40.495123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195797f3-599f-4770-8ef2-afbc01ce32f0 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.506134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195797f3-599f-4770-8ef2-afbc01ce32f0 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.511109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f5fd5-1642-4fb7-bc08-1fff569d66e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.511705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f5fd5-1642-4fb7-bc08-1fff569d66e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.511881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 911/1846] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:40 2022] POST /identity/v3/users => generated 282 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:40.511986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f5fd5-1642-4fb7-bc08-1fff569d66e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.512255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f5fd5-1642-4fb7-bc08-1fff569d66e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.512722 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c91b6-3722-4267-927c-6a8d5f18af8e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.512866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f5fd5-1642-4fb7-bc08-1fff569d66e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.514983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f5fd5-1642-4fb7-bc08-1fff569d66e1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.526861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 936/1847] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:40.527592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f5fd5-1642-4fb7-bc08-1fff569d66e1 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.530716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c91b6-3722-4267-927c-6a8d5f18af8e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.535937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249ddbe-5dd2-4492-8f49-4687693b2b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.536930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249ddbe-5dd2-4492-8f49-4687693b2b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.536930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249ddbe-5dd2-4492-8f49-4687693b2b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.537306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249ddbe-5dd2-4492-8f49-4687693b2b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.538033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249ddbe-5dd2-4492-8f49-4687693b2b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.539879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249ddbe-5dd2-4492-8f49-4687693b2b01 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.540841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f5fd5-1642-4fb7-bc08-1fff569d66e1 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.547839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d36aa-829f-4d9e-84da-2b170b3242b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.548666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d36aa-829f-4d9e-84da-2b170b3242b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.549013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d36aa-829f-4d9e-84da-2b170b3242b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.549319 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d36aa-829f-4d9e-84da-2b170b3242b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.549965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d36aa-829f-4d9e-84da-2b170b3242b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.551673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48d36aa-829f-4d9e-84da-2b170b3242b2 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.553519 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 912/1848] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:40.557670 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249ddbe-5dd2-4492-8f49-4687693b2b01 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.566849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 937/1849] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:40.566849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d36aa-829f-4d9e-84da-2b170b3242b2 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.567717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249ddbe-5dd2-4492-8f49-4687693b2b01 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.573267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26aae8-ea25-4012-a182-7e74d3a5b7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.573884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26aae8-ea25-4012-a182-7e74d3a5b7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.574161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26aae8-ea25-4012-a182-7e74d3a5b7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.574422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26aae8-ea25-4012-a182-7e74d3a5b7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.575063 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26aae8-ea25-4012-a182-7e74d3a5b7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.576078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d36aa-829f-4d9e-84da-2b170b3242b2 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.577234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26aae8-ea25-4012-a182-7e74d3a5b7b0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.581423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12397d-3d24-4dcc-843b-7ae04a742fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.582130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12397d-3d24-4dcc-843b-7ae04a742fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.582463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12397d-3d24-4dcc-843b-7ae04a742fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.583093 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12397d-3d24-4dcc-843b-7ae04a742fb5 admin admin] PATH_INFO: `/v3/projects/cd3e119c3abc473786626a27c865300e/users/ec01a5f046e24da5b917fbecdc05a2f8/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.593556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 913/1850] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:40.593556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26aae8-ea25-4012-a182-7e74d3a5b7b0 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.602280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12397d-3d24-4dcc-843b-7ae04a742fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3e119c3abc473786626a27c865300e, user_id=ec01a5f046e24da5b917fbecdc05a2f8, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.603671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26aae8-ea25-4012-a182-7e74d3a5b7b0 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.605058 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12397d-3d24-4dcc-843b-7ae04a742fb5 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.608909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f698ec5-27cb-48e5-a7df-88aafb767f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.609633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f698ec5-27cb-48e5-a7df-88aafb767f56 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.610020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f698ec5-27cb-48e5-a7df-88aafb767f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.610165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f698ec5-27cb-48e5-a7df-88aafb767f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.610907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f698ec5-27cb-48e5-a7df-88aafb767f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.612462 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f698ec5-27cb-48e5-a7df-88aafb767f56 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.620586 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 938/1851] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:40 2022] PUT /identity/v3/projects/cd3e119c3abc473786626a27c865300e/users/ec01a5f046e24da5b917fbecdc05a2f8/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:40.621588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12397d-3d24-4dcc-843b-7ae04a742fb5 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.624765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 914/1852] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:40.628662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f698ec5-27cb-48e5-a7df-88aafb767f56 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.639653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f698ec5-27cb-48e5-a7df-88aafb767f56 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.659069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:40.697491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12397d-3d24-4dcc-843b-7ae04a742fb5 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.703003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c4ce9-e61c-406b-93f4-e06ef94b9145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.703665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c4ce9-e61c-406b-93f4-e06ef94b9145 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.703941 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c4ce9-e61c-406b-93f4-e06ef94b9145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.704204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c4ce9-e61c-406b-93f4-e06ef94b9145 admin admin] PATH_INFO: `/v3/projects/1c13802cf7184b28a757f212a45b06d9/users/86ff90ffcc0c4b73af1c6454da3e549a/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.704392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e182a-933d-479a-8474-599c08217661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.705054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e182a-933d-479a-8474-599c08217661 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.705372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e182a-933d-479a-8474-599c08217661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.705642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e182a-933d-479a-8474-599c08217661 admin admin] PATH_INFO: `/v3/projects/8aeb59d96a8f4eebae8478e30dbc865f/users/18f57ae1a90847579c5d3fff565dbbc1/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.720324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c4ce9-e61c-406b-93f4-e06ef94b9145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c13802cf7184b28a757f212a45b06d9, user_id=86ff90ffcc0c4b73af1c6454da3e549a, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.722601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e182a-933d-479a-8474-599c08217661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aeb59d96a8f4eebae8478e30dbc865f, user_id=18f57ae1a90847579c5d3fff565dbbc1, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.723003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c4ce9-e61c-406b-93f4-e06ef94b9145 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.725274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e182a-933d-479a-8474-599c08217661 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:40.725528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:40.739473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 939/1853] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:40 2022] PUT /identity/v3/projects/1c13802cf7184b28a757f212a45b06d9/users/86ff90ffcc0c4b73af1c6454da3e549a/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:40.750398 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 915/1854] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:40 2022] PUT /identity/v3/projects/8aeb59d96a8f4eebae8478e30dbc865f/users/18f57ae1a90847579c5d3fff565dbbc1/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:38:40.750585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c618914-f95f-493c-a795-1b2ec61ad1b5 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.750585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c618914-f95f-493c-a795-1b2ec61ad1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.750585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c618914-f95f-493c-a795-1b2ec61ad1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.751890 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c86a59-701e-4e5d-95b6-ade5dfb6978f None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.752204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c86a59-701e-4e5d-95b6-ade5dfb6978f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.752619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c86a59-701e-4e5d-95b6-ade5dfb6978f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.774522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5c618914-f95f-493c-a795-1b2ec61ad1b5 None None] MFA Rules not processed for user `ec01a5f046e24da5b917fbecdc05a2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:40.775766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5c618914-f95f-493c-a795-1b2ec61ad1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:40.782411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-46c86a59-701e-4e5d-95b6-ade5dfb6978f None None] MFA Rules not processed for user `86ff90ffcc0c4b73af1c6454da3e549a`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:40.783959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-46c86a59-701e-4e5d-95b6-ade5dfb6978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:40.846072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 940/1855] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:40.847108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4ffd9-f67c-4ce0-ac1b-d9bf68b7bf0a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.847590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4ffd9-f67c-4ce0-ac1b-d9bf68b7bf0a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.847837 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4ffd9-f67c-4ce0-ac1b-d9bf68b7bf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.856982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 916/1856] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:40.857871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c86a59-701e-4e5d-95b6-ade5dfb6978f None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.859156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-46c86a59-701e-4e5d-95b6-ade5dfb6978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:40.883112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-7fc4ffd9-f67c-4ce0-ac1b-d9bf68b7bf0a None None] MFA Rules not processed for user `18f57ae1a90847579c5d3fff565dbbc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:40.884318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7fc4ffd9-f67c-4ce0-ac1b-d9bf68b7bf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:40.962603 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c86a59-701e-4e5d-95b6-ade5dfb6978f None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:40.973776 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d37352-a617-4e09-92b3-b8bec38d5a78 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff22c2632c614fca8008feab93c8fa2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:40.974176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 941/1857] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:40.974409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d37352-a617-4e09-92b3-b8bec38d5a78 service glance] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:40.974691 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d37352-a617-4e09-92b3-b8bec38d5a78 service glance] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:40.975003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d37352-a617-4e09-92b3-b8bec38d5a78 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:40.975108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4ffd9-f67c-4ce0-ac1b-d9bf68b7bf0a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:40.993144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d37352-a617-4e09-92b3-b8bec38d5a78 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:40.995086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d37352-a617-4e09-92b3-b8bec38d5a78 service glance] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.008577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 917/1858] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:41.009968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d37352-a617-4e09-92b3-b8bec38d5a78 service glance] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:41.039136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:41.069903 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4ffd9-f67c-4ce0-ac1b-d9bf68b7bf0a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:41.076187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d367e-f3d1-47a7-9179-0e457baaeeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:41.077138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d367e-f3d1-47a7-9179-0e457baaeeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.077512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d367e-f3d1-47a7-9179-0e457baaeeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.078022 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d367e-f3d1-47a7-9179-0e457baaeeb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.079153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d367e-f3d1-47a7-9179-0e457baaeeb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:41.081554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d367e-f3d1-47a7-9179-0e457baaeeb2 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.094655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d37352-a617-4e09-92b3-b8bec38d5a78 service glance] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:41.099633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cae6a-4bfa-4bef-90fc-a6bb8c708387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:41.100179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cae6a-4bfa-4bef-90fc-a6bb8c708387 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.101427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cae6a-4bfa-4bef-90fc-a6bb8c708387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.102397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cae6a-4bfa-4bef-90fc-a6bb8c708387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.103363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cae6a-4bfa-4bef-90fc-a6bb8c708387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:41.106111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cae6a-4bfa-4bef-90fc-a6bb8c708387 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.110184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:41.138989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 942/1859] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:40 2022] POST /identity/v3/projects => generated 369 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:41.142891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d367e-f3d1-47a7-9179-0e457baaeeb2 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:41.149715 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 918/1860] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:41 2022] POST /identity/v3/projects => generated 363 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:41.153783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cae6a-4bfa-4bef-90fc-a6bb8c708387 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:41.243215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cae6a-4bfa-4bef-90fc-a6bb8c708387 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:41.243357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d367e-f3d1-47a7-9179-0e457baaeeb2 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:41.249158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0ccfc-6fdb-498e-bcda-3f2891e4be6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:41.249833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0ccfc-6fdb-498e-bcda-3f2891e4be6b admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.250115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0ccfc-6fdb-498e-bcda-3f2891e4be6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.250387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0ccfc-6fdb-498e-bcda-3f2891e4be6b admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.251121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0ccfc-6fdb-498e-bcda-3f2891e4be6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:41.251382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37d8a9-7893-49cf-b0b0-81c63177e209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:41.252760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37d8a9-7893-49cf-b0b0-81c63177e209 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.253016 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0ccfc-6fdb-498e-bcda-3f2891e4be6b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.253212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37d8a9-7893-49cf-b0b0-81c63177e209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.253597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37d8a9-7893-49cf-b0b0-81c63177e209 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.254290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37d8a9-7893-49cf-b0b0-81c63177e209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:41.256311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37d8a9-7893-49cf-b0b0-81c63177e209 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.286253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 919/1861] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:41 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:38:41.290837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37d8a9-7893-49cf-b0b0-81c63177e209 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:41.300032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 943/1862] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:41 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:41.302313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37d8a9-7893-49cf-b0b0-81c63177e209 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:41.304529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0ccfc-6fdb-498e-bcda-3f2891e4be6b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:41.309390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048b419-96c0-4990-9ed1-ec36077f5d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:41.310878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048b419-96c0-4990-9ed1-ec36077f5d22 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.311169 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048b419-96c0-4990-9ed1-ec36077f5d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.312166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048b419-96c0-4990-9ed1-ec36077f5d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.312877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048b419-96c0-4990-9ed1-ec36077f5d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:41.315515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048b419-96c0-4990-9ed1-ec36077f5d22 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.317833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0ccfc-6fdb-498e-bcda-3f2891e4be6b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:41.323607 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ec9bf-3e95-4b0c-af37-34b9ae3f70ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:41.324448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ec9bf-3e95-4b0c-af37-34b9ae3f70ee admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.324877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ec9bf-3e95-4b0c-af37-34b9ae3f70ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.325364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ec9bf-3e95-4b0c-af37-34b9ae3f70ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.326247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ec9bf-3e95-4b0c-af37-34b9ae3f70ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:41.328256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ec9bf-3e95-4b0c-af37-34b9ae3f70ee admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.328471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 920/1863] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:38:41.333431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048b419-96c0-4990-9ed1-ec36077f5d22 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:41.345336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048b419-96c0-4990-9ed1-ec36077f5d22 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:41.350279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4ee4-a0e1-4a43-8966-73ab5747e340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:41.350906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4ee4-a0e1-4a43-8966-73ab5747e340 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.351080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4ee4-a0e1-4a43-8966-73ab5747e340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.351597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef4ee4-a0e1-4a43-8966-73ab5747e340 admin admin] PATH_INFO: `/v3/projects/8630e75ed37e4ccb989b500c6c44a0fd/users/46ef97e1a3554bea989da45a726d5c61/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.355563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 944/1864] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:41.357535 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ec9bf-3e95-4b0c-af37-34b9ae3f70ee admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:41.372982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef4ee4-a0e1-4a43-8966-73ab5747e340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8630e75ed37e4ccb989b500c6c44a0fd, user_id=46ef97e1a3554bea989da45a726d5c61, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:41.373531 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ec9bf-3e95-4b0c-af37-34b9ae3f70ee admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:41.376693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef4ee4-a0e1-4a43-8966-73ab5747e340 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.383485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9599e-9d26-48f2-9f21-cef66b6b50ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:41.386304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9599e-9d26-48f2-9f21-cef66b6b50ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.386720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9599e-9d26-48f2-9f21-cef66b6b50ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.387542 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9599e-9d26-48f2-9f21-cef66b6b50ea admin admin] PATH_INFO: `/v3/projects/863cf2fb37814870b2b9ccd93efceda1/users/6e8ce8579a934cf58a0c5d03d963304e/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.392752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 921/1865] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:41 2022] PUT /identity/v3/projects/8630e75ed37e4ccb989b500c6c44a0fd/users/46ef97e1a3554bea989da45a726d5c61/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:38:41.396840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4ee4-a0e1-4a43-8966-73ab5747e340 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:41.410205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9599e-9d26-48f2-9f21-cef66b6b50ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=863cf2fb37814870b2b9ccd93efceda1, user_id=6e8ce8579a934cf58a0c5d03d963304e, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:41.432718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9599e-9d26-48f2-9f21-cef66b6b50ea admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.451650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 945/1866] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:41 2022] PUT /identity/v3/projects/863cf2fb37814870b2b9ccd93efceda1/users/6e8ce8579a934cf58a0c5d03d963304e/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:41.455745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9599e-9d26-48f2-9f21-cef66b6b50ea admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:41.565373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:41.575607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef4ee4-a0e1-4a43-8966-73ab5747e340 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:41.610036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9599e-9d26-48f2-9f21-cef66b6b50ea admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:41.622747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092661c0-f87a-443c-aec8-f257769a8cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:41.624533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092661c0-f87a-443c-aec8-f257769a8cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.624926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092661c0-f87a-443c-aec8-f257769a8cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.625338 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092661c0-f87a-443c-aec8-f257769a8cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.626701 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092661c0-f87a-443c-aec8-f257769a8cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:41.628530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092661c0-f87a-443c-aec8-f257769a8cf6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.657133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 946/1867] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:38:41.662246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092661c0-f87a-443c-aec8-f257769a8cf6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:41.681026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092661c0-f87a-443c-aec8-f257769a8cf6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:41.687285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df07e831-da9f-4baf-85d9-6b350d42c52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:41.688337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07e831-da9f-4baf-85d9-6b350d42c52a admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.688679 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07e831-da9f-4baf-85d9-6b350d42c52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.689530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df07e831-da9f-4baf-85d9-6b350d42c52a admin admin] PATH_INFO: `/v3/projects/863cf2fb37814870b2b9ccd93efceda1/users/6e8ce8579a934cf58a0c5d03d963304e/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.691858 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df07e831-da9f-4baf-85d9-6b350d42c52a admin admin] RBAC: Authorizing `identity:create_grant(project_id=863cf2fb37814870b2b9ccd93efceda1, user_id=6e8ce8579a934cf58a0c5d03d963304e, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:41.694940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df07e831-da9f-4baf-85d9-6b350d42c52a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.706856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4336c02-89e0-464f-b985-a92fda6cc143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:41.707691 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4336c02-89e0-464f-b985-a92fda6cc143 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.707971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4336c02-89e0-464f-b985-a92fda6cc143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.708245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4336c02-89e0-464f-b985-a92fda6cc143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.708903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4336c02-89e0-464f-b985-a92fda6cc143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:41.711147 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4336c02-89e0-464f-b985-a92fda6cc143 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.724073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 947/1868] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:41 2022] PUT /identity/v3/projects/863cf2fb37814870b2b9ccd93efceda1/users/6e8ce8579a934cf58a0c5d03d963304e/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:41.735544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33463d-cc40-40b3-b99c-210c26903765 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.735544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33463d-cc40-40b3-b99c-210c26903765 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.736138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33463d-cc40-40b3-b99c-210c26903765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.745259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 922/1869] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:41.746571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4336c02-89e0-464f-b985-a92fda6cc143 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:41.764121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4336c02-89e0-464f-b985-a92fda6cc143 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:41.792817 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-4b33463d-cc40-40b3-b99c-210c26903765 None None] MFA Rules not processed for user `6e8ce8579a934cf58a0c5d03d963304e`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:41.794299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4b33463d-cc40-40b3-b99c-210c26903765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:41.823751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:41.871874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b68d9-7aa5-4b51-8f64-302cb4f4f2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:41.873606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b68d9-7aa5-4b51-8f64-302cb4f4f2f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.874145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b68d9-7aa5-4b51-8f64-302cb4f4f2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.875135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b68d9-7aa5-4b51-8f64-302cb4f4f2f5 admin admin] PATH_INFO: `/v3/projects/8630e75ed37e4ccb989b500c6c44a0fd/users/46ef97e1a3554bea989da45a726d5c61/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.878287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b68d9-7aa5-4b51-8f64-302cb4f4f2f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8630e75ed37e4ccb989b500c6c44a0fd, user_id=46ef97e1a3554bea989da45a726d5c61, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:41.880467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b68d9-7aa5-4b51-8f64-302cb4f4f2f5 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:41.893864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 923/1870] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:41 2022] PUT /identity/v3/projects/8630e75ed37e4ccb989b500c6c44a0fd/users/46ef97e1a3554bea989da45a726d5c61/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:41.907069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 948/1871] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:38:41.907279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de4dd9-3b55-4582-9e2c-26e0cc8c7abd None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:41.907279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de4dd9-3b55-4582-9e2c-26e0cc8c7abd None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:41.907279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de4dd9-3b55-4582-9e2c-26e0cc8c7abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:41.915903 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33463d-cc40-40b3-b99c-210c26903765 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:41.918168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4b33463d-cc40-40b3-b99c-210c26903765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:41.947972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-35de4dd9-3b55-4582-9e2c-26e0cc8c7abd None None] MFA Rules not processed for user `46ef97e1a3554bea989da45a726d5c61`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:41.950175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-35de4dd9-3b55-4582-9e2c-26e0cc8c7abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:41.991406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33463d-cc40-40b3-b99c-210c26903765 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:42.055964 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 924/1872] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:38:42.071652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de4dd9-3b55-4582-9e2c-26e0cc8c7abd None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:42.073776 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-35de4dd9-3b55-4582-9e2c-26e0cc8c7abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:42.087615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fed69-d6ee-49c7-8573-5f29be3f71a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:42.089359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fed69-d6ee-49c7-8573-5f29be3f71a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:42.090195 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fed69-d6ee-49c7-8573-5f29be3f71a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:42.090195 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fed69-d6ee-49c7-8573-5f29be3f71a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:42.167363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:42.232848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fed69-d6ee-49c7-8573-5f29be3f71a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:42.234549 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fed69-d6ee-49c7-8573-5f29be3f71a8 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:42.250579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 949/1873] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:38:42.278672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de4dd9-3b55-4582-9e2c-26e0cc8c7abd None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:42.291656 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930ca14-185e-47e0-bd3d-f965f4e47320 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:42.295052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930ca14-185e-47e0-bd3d-f965f4e47320 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:42.295351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930ca14-185e-47e0-bd3d-f965f4e47320 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:42.295670 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930ca14-185e-47e0-bd3d-f965f4e47320 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:42.321426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930ca14-185e-47e0-bd3d-f965f4e47320 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:42.323057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930ca14-185e-47e0-bd3d-f965f4e47320 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:42.349000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 925/1874] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:42.526664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fed69-d6ee-49c7-8573-5f29be3f71a8 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:42.540268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fed69-d6ee-49c7-8573-5f29be3f71a8 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:42.553039 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1448401-9a5e-4762-aab7-b898b0852fa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:42.554271 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1448401-9a5e-4762-aab7-b898b0852fa0 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:42.554760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1448401-9a5e-4762-aab7-b898b0852fa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:42.555158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1448401-9a5e-4762-aab7-b898b0852fa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:42.636464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:42.709874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1448401-9a5e-4762-aab7-b898b0852fa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:42.712979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1448401-9a5e-4762-aab7-b898b0852fa0 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:42.751821 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 950/1875] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:44.245330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930ca14-185e-47e0-bd3d-f965f4e47320 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:44.319299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930ca14-185e-47e0-bd3d-f965f4e47320 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:44.324639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:44.325327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:44.326139 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:44.326423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] PATH_INFO: `/v3/users/86ff90ffcc0c4b73af1c6454da3e549a` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:44.327567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=86ff90ffcc0c4b73af1c6454da3e549a)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:44.329553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:44.385423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ff90ffcc0c4b73af1c6454da3e549a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:44.396366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ff90ffcc0c4b73af1c6454da3e549a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:44.397294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ff90ffcc0c4b73af1c6454da3e549a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:44.397903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ff90ffcc0c4b73af1c6454da3e549a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:44.407133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ff90ffcc0c4b73af1c6454da3e549a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:44.425739 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 926/1876] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:44 2022] DELETE /identity/v3/users/86ff90ffcc0c4b73af1c6454da3e549a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:44.430384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724eef-eb92-4fa8-9636-1284686bc379 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:44.430892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724eef-eb92-4fa8-9636-1284686bc379 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:44.431017 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724eef-eb92-4fa8-9636-1284686bc379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:44.451582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-aa724eef-eb92-4fa8-9636-1284686bc379 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:44.452843 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-aa724eef-eb92-4fa8-9636-1284686bc379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:44.514568 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 951/1877] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:44.524869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:44.526574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:44.575364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:44.595784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768aa20c-9d85-43ff-814f-1c8d9a875a6b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:44.601668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa793da-0a54-4dd2-aedd-fdb3b279d8d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:44.602438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa793da-0a54-4dd2-aedd-fdb3b279d8d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:44.602721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa793da-0a54-4dd2-aedd-fdb3b279d8d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:44.602995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa793da-0a54-4dd2-aedd-fdb3b279d8d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:44.612714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa793da-0a54-4dd2-aedd-fdb3b279d8d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:44.614197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa793da-0a54-4dd2-aedd-fdb3b279d8d5 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:44.624642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 927/1878] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:38:44.906666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa724eef-eb92-4fa8-9636-1284686bc379 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:44.908186 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-aa724eef-eb92-4fa8-9636-1284686bc379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:44.923727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa724eef-eb92-4fa8-9636-1284686bc379 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:44.929704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:44.930383 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:44.930662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:44.931007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] PATH_INFO: `/v3/projects/1c13802cf7184b28a757f212a45b06d9` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:44.932231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c13802cf7184b28a757f212a45b06d9)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:44.934150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:44.984505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c13802cf7184b28a757f212a45b06d9', 'request_id': 'req-b4aeb9d0-1734-496a-9839-069f8f6c8be6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:44.990159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c13802cf7184b28a757f212a45b06d9', 'request_id': 'req-b4aeb9d0-1734-496a-9839-069f8f6c8be6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:44.996419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c13802cf7184b28a757f212a45b06d9', 'request_id': 'req-b4aeb9d0-1734-496a-9839-069f8f6c8be6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.011951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] The token cache is being invalidate because project 1c13802cf7184b28a757f212a45b06d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:45.012246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.012904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.014483 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 952/1879] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:44 2022] DELETE /identity/v3/projects/1c13802cf7184b28a757f212a45b06d9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:45.018558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa793da-0a54-4dd2-aedd-fdb3b279d8d5 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:45.028222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa793da-0a54-4dd2-aedd-fdb3b279d8d5 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:45.095255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c63b5-8115-404e-86b2-f11dbf4b4901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:45.095905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c63b5-8115-404e-86b2-f11dbf4b4901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:45.096189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c63b5-8115-404e-86b2-f11dbf4b4901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:45.096460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c63b5-8115-404e-86b2-f11dbf4b4901 admin admin] PATH_INFO: `/v3/users/6e8ce8579a934cf58a0c5d03d963304e` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:45.097669 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c63b5-8115-404e-86b2-f11dbf4b4901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8ce8579a934cf58a0c5d03d963304e)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:45.101595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c63b5-8115-404e-86b2-f11dbf4b4901 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:45.176713 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-465c63b5-8115-404e-86b2-f11dbf4b4901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8ce8579a934cf58a0c5d03d963304e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.191801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-465c63b5-8115-404e-86b2-f11dbf4b4901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8ce8579a934cf58a0c5d03d963304e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.191801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-465c63b5-8115-404e-86b2-f11dbf4b4901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8ce8579a934cf58a0c5d03d963304e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.191801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-465c63b5-8115-404e-86b2-f11dbf4b4901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8ce8579a934cf58a0c5d03d963304e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.206018 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-465c63b5-8115-404e-86b2-f11dbf4b4901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8ce8579a934cf58a0c5d03d963304e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.239251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 928/1880] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:38:45 2022] DELETE /identity/v3/users/6e8ce8579a934cf58a0c5d03d963304e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:45.725000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:45.726645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:45.742038 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aeb9d0-1734-496a-9839-069f8f6c8be6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:45.791332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:45.806771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:45.807448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:45.807732 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:45.808005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f admin admin] PATH_INFO: `/v3/projects/863cf2fb37814870b2b9ccd93efceda1` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:45.809226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=863cf2fb37814870b2b9ccd93efceda1)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:45.811775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:45.863703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '863cf2fb37814870b2b9ccd93efceda1', 'request_id': 'req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.869627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '863cf2fb37814870b2b9ccd93efceda1', 'request_id': 'req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.877316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '863cf2fb37814870b2b9ccd93efceda1', 'request_id': 'req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.892199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f admin admin] The token cache is being invalidate because project 863cf2fb37814870b2b9ccd93efceda1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:38:45.892490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.893370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-1ca11ab7-2e8d-4180-bd81-e5a078d02f7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:38:45.895248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 953/1881] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:38:45 2022] DELETE /identity/v3/projects/863cf2fb37814870b2b9ccd93efceda1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:45.901310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aed6d-4a36-4a14-afac-4d3c7712ce8a None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:45.901868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aed6d-4a36-4a14-afac-4d3c7712ce8a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:45.901961 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aed6d-4a36-4a14-afac-4d3c7712ce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:45.923317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-eb4aed6d-4a36-4a14-afac-4d3c7712ce8a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:45.924543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-eb4aed6d-4a36-4a14-afac-4d3c7712ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:45.986761 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 929/1882] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:45.995150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83072b27-84d3-44cc-ad55-c42fc3d8c49e None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:45.997287 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83072b27-84d3-44cc-ad55-c42fc3d8c49e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:45.997632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83072b27-84d3-44cc-ad55-c42fc3d8c49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:46.017518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-83072b27-84d3-44cc-ad55-c42fc3d8c49e None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:46.018643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-83072b27-84d3-44cc-ad55-c42fc3d8c49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:46.026519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 954/1883] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:46.031393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4aed6d-4a36-4a14-afac-4d3c7712ce8a None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:46.037716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:46.042125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4aed6d-4a36-4a14-afac-4d3c7712ce8a None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:46.049306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a682d-bad0-4908-bac9-291348b0185d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:46.050058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a682d-bad0-4908-bac9-291348b0185d admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:46.050517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a682d-bad0-4908-bac9-291348b0185d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:46.050912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a682d-bad0-4908-bac9-291348b0185d admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:46.052015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a682d-bad0-4908-bac9-291348b0185d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:38:46.052500 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a682d-bad0-4908-bac9-291348b0185d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:46.055095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a682d-bad0-4908-bac9-291348b0185d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:46.067635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 930/1884] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:38:46.072056 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83072b27-84d3-44cc-ad55-c42fc3d8c49e None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:46.081758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83072b27-84d3-44cc-ad55-c42fc3d8c49e None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:46.086380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b65b0-8ae1-4dbd-95e2-ab743df78532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:46.086984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b65b0-8ae1-4dbd-95e2-ab743df78532 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:46.087260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b65b0-8ae1-4dbd-95e2-ab743df78532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:46.087529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b65b0-8ae1-4dbd-95e2-ab743df78532 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:46.088162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b65b0-8ae1-4dbd-95e2-ab743df78532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:46.090107 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b65b0-8ae1-4dbd-95e2-ab743df78532 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:46.124621 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 955/1885] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:38:46 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:46.125813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a682d-bad0-4908-bac9-291348b0185d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:46.135632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a682d-bad0-4908-bac9-291348b0185d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:46.202839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0067948-3605-49d7-8b40-4cd1ab66b96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:46.203483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0067948-3605-49d7-8b40-4cd1ab66b96e admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:46.203765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0067948-3605-49d7-8b40-4cd1ab66b96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:46.204035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0067948-3605-49d7-8b40-4cd1ab66b96e admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:46.204705 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0067948-3605-49d7-8b40-4cd1ab66b96e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:46.206724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0067948-3605-49d7-8b40-4cd1ab66b96e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:46.231145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 931/1886] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:38:46 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:38:46.238046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b65b0-8ae1-4dbd-95e2-ab743df78532 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:46.252705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b65b0-8ae1-4dbd-95e2-ab743df78532 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:46.259432 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f75dda-b38d-4c3a-b567-f43cc4753b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:46.260059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f75dda-b38d-4c3a-b567-f43cc4753b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:46.260346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f75dda-b38d-4c3a-b567-f43cc4753b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:46.260623 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f75dda-b38d-4c3a-b567-f43cc4753b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:46.261316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f75dda-b38d-4c3a-b567-f43cc4753b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:46.263443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f75dda-b38d-4c3a-b567-f43cc4753b4e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:46.275121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 956/1887] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:38:46.280118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0067948-3605-49d7-8b40-4cd1ab66b96e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:46.293746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0067948-3605-49d7-8b40-4cd1ab66b96e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:46.299766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf2205-75f7-46da-a58f-c587b385e416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:46.300474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf2205-75f7-46da-a58f-c587b385e416 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:46.300760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf2205-75f7-46da-a58f-c587b385e416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:46.301198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf2205-75f7-46da-a58f-c587b385e416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:46.301974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf2205-75f7-46da-a58f-c587b385e416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:46.304109 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf2205-75f7-46da-a58f-c587b385e416 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:46.316101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 932/1888] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:38:46.320135 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f75dda-b38d-4c3a-b567-f43cc4753b4e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:46.333397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f75dda-b38d-4c3a-b567-f43cc4753b4e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:46.339234 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e21bc-cec7-4e51-a837-aaa7f8192b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:46.339914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e21bc-cec7-4e51-a837-aaa7f8192b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:46.340193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e21bc-cec7-4e51-a837-aaa7f8192b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:46.340645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e21bc-cec7-4e51-a837-aaa7f8192b9e admin admin] PATH_INFO: `/v3/projects/a40205da07f242a48c7ee37350a1d3bf/users/a32aa66a7b7746f998aec78df1b5cbef/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:46.359600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e21bc-cec7-4e51-a837-aaa7f8192b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a40205da07f242a48c7ee37350a1d3bf, user_id=a32aa66a7b7746f998aec78df1b5cbef, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:46.362383 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e21bc-cec7-4e51-a837-aaa7f8192b9e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:46.377301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 957/1889] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:38:46 2022] PUT /identity/v3/projects/a40205da07f242a48c7ee37350a1d3bf/users/a32aa66a7b7746f998aec78df1b5cbef/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:38:46.389549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec58194-f41a-4cf8-8c65-789028019926 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:46.389903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec58194-f41a-4cf8-8c65-789028019926 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:46.390088 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec58194-f41a-4cf8-8c65-789028019926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:46.428860 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-aec58194-f41a-4cf8-8c65-789028019926 None None] MFA Rules not processed for user `a32aa66a7b7746f998aec78df1b5cbef`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:38:46.430332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-aec58194-f41a-4cf8-8c65-789028019926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:46.556627 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 933/1890] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:38:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:38:46.569623 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e21bc-cec7-4e51-a837-aaa7f8192b9e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:38:46.571497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-625e21bc-cec7-4e51-a837-aaa7f8192b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:38:46.658283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e21bc-cec7-4e51-a837-aaa7f8192b9e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:38:46.662660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f79be-3c3c-4329-8284-311a7a52fcce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff22c2632c614fca8008feab93c8fa2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:38:46.663290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f79be-3c3c-4329-8284-311a7a52fcce service glance] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:38:46.663573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f79be-3c3c-4329-8284-311a7a52fcce service glance] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:38:46.663861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f79be-3c3c-4329-8284-311a7a52fcce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:38:46.675750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f79be-3c3c-4329-8284-311a7a52fcce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:38:46.677365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f79be-3c3c-4329-8284-311a7a52fcce service glance] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:38:46.691869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 958/1891] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:38:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:38:56.552469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:38:56.683467 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:06.749200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d60951-172e-471b-a807-0cf14a59c956 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:06.749680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d60951-172e-471b-a807-0cf14a59c956 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:06.749921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d60951-172e-471b-a807-0cf14a59c956 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:06.751527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 934/1892] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:39:06.771620 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c30baf-fa64-4ae3-8470-f05ad554bae0 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:06.773245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c30baf-fa64-4ae3-8470-f05ad554bae0 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:06.773771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c30baf-fa64-4ae3-8470-f05ad554bae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:06.800135 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-32c30baf-fa64-4ae3-8470-f05ad554bae0 None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:06.801833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-32c30baf-fa64-4ae3-8470-f05ad554bae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:06.860478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d60951-172e-471b-a807-0cf14a59c956 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:06.862201 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-54d60951-172e-471b-a807-0cf14a59c956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:06.885360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 959/1893] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:39:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:06.897660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c30baf-fa64-4ae3-8470-f05ad554bae0 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:06.898995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-32c30baf-fa64-4ae3-8470-f05ad554bae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:06.913024 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c30baf-fa64-4ae3-8470-f05ad554bae0 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:06.919125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab6a5d-d0ee-430e-880d-27fc7c48f519 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:06.919827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab6a5d-d0ee-430e-880d-27fc7c48f519 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:06.920413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab6a5d-d0ee-430e-880d-27fc7c48f519 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:06.920720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab6a5d-d0ee-430e-880d-27fc7c48f519 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:06.929712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d60951-172e-471b-a807-0cf14a59c956 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:06.931007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab6a5d-d0ee-430e-880d-27fc7c48f519 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:06.932729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab6a5d-d0ee-430e-880d-27fc7c48f519 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:06.934164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:06.934765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:06.935045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:06.935314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] PATH_INFO: `/v3/users/18f57ae1a90847579c5d3fff565dbbc1` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:06.936476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] RBAC: Authorizing `identity:delete_user(user_id=18f57ae1a90847579c5d3fff565dbbc1)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:06.938406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:06.943678 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 960/1894] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:39:06.995184 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18f57ae1a90847579c5d3fff565dbbc1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:07.002481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18f57ae1a90847579c5d3fff565dbbc1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:07.003213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18f57ae1a90847579c5d3fff565dbbc1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:07.003823 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18f57ae1a90847579c5d3fff565dbbc1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:07.011711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18f57ae1a90847579c5d3fff565dbbc1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:07.030979 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 935/1895] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:06 2022] DELETE /identity/v3/users/18f57ae1a90847579c5d3fff565dbbc1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:39:07.035859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57baae6-6a25-41aa-b840-6d8573d9ff11 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:07.036226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57baae6-6a25-41aa-b840-6d8573d9ff11 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:07.036576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57baae6-6a25-41aa-b840-6d8573d9ff11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:07.056653 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e57baae6-6a25-41aa-b840-6d8573d9ff11 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:07.057912 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e57baae6-6a25-41aa-b840-6d8573d9ff11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:07.124720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 961/1896] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:07.134920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:07.136372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:07.227479 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7068058-7f3c-4f2f-815f-b4df88b414af admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:07.231451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c2309-fd51-4b88-bffe-628b062a29ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:07.232170 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c2309-fd51-4b88-bffe-628b062a29ea service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:07.232513 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c2309-fd51-4b88-bffe-628b062a29ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:07.232849 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c2309-fd51-4b88-bffe-628b062a29ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:07.243660 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c2309-fd51-4b88-bffe-628b062a29ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:07.245654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c2309-fd51-4b88-bffe-628b062a29ea service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:07.257288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 936/1897] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:39:08.043789 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57baae6-6a25-41aa-b840-6d8573d9ff11 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:08.048497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e57baae6-6a25-41aa-b840-6d8573d9ff11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:08.071654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57baae6-6a25-41aa-b840-6d8573d9ff11 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:08.076760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:08.077713 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:08.078002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:08.078277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] PATH_INFO: `/v3/projects/8aeb59d96a8f4eebae8478e30dbc865f` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:08.079475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aeb59d96a8f4eebae8478e30dbc865f)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:08.081746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:08.139268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aeb59d96a8f4eebae8478e30dbc865f', 'request_id': 'req-bc9db24b-ff96-429e-8ea2-47c80004cc2f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.145132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aeb59d96a8f4eebae8478e30dbc865f', 'request_id': 'req-bc9db24b-ff96-429e-8ea2-47c80004cc2f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.152833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aeb59d96a8f4eebae8478e30dbc865f', 'request_id': 'req-bc9db24b-ff96-429e-8ea2-47c80004cc2f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.167657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] The token cache is being invalidate because project 8aeb59d96a8f4eebae8478e30dbc865f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:08.167945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.168590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.170157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 962/1898] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:08 2022] DELETE /identity/v3/projects/8aeb59d96a8f4eebae8478e30dbc865f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:08.174724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c2309-fd51-4b88-bffe-628b062a29ea service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:08.185343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c2309-fd51-4b88-bffe-628b062a29ea service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:08.206516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:08.264596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb448b5-f2cc-4a6a-89dc-55c7153af593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:08.265258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb448b5-f2cc-4a6a-89dc-55c7153af593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:08.265528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb448b5-f2cc-4a6a-89dc-55c7153af593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:08.265800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb448b5-f2cc-4a6a-89dc-55c7153af593 admin admin] PATH_INFO: `/v3/users/46ef97e1a3554bea989da45a726d5c61` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:08.267095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb448b5-f2cc-4a6a-89dc-55c7153af593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46ef97e1a3554bea989da45a726d5c61)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:08.269194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb448b5-f2cc-4a6a-89dc-55c7153af593 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:08.328387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8cb448b5-f2cc-4a6a-89dc-55c7153af593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46ef97e1a3554bea989da45a726d5c61'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.336317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8cb448b5-f2cc-4a6a-89dc-55c7153af593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46ef97e1a3554bea989da45a726d5c61'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.337029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8cb448b5-f2cc-4a6a-89dc-55c7153af593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46ef97e1a3554bea989da45a726d5c61'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.337671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8cb448b5-f2cc-4a6a-89dc-55c7153af593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46ef97e1a3554bea989da45a726d5c61'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.347802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8cb448b5-f2cc-4a6a-89dc-55c7153af593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46ef97e1a3554bea989da45a726d5c61'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.370512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 937/1899] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:08 2022] DELETE /identity/v3/users/46ef97e1a3554bea989da45a726d5c61 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:08.770258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:08.771769 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:08.782601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:08.790713 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9db24b-ff96-429e-8ea2-47c80004cc2f admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:08.862283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cfe26-ec34-46e5-9015-224bd2042f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:08.863046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cfe26-ec34-46e5-9015-224bd2042f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:08.863330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cfe26-ec34-46e5-9015-224bd2042f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:08.863606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cfe26-ec34-46e5-9015-224bd2042f15 admin admin] PATH_INFO: `/v3/projects/8630e75ed37e4ccb989b500c6c44a0fd` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:08.864789 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3cfe26-ec34-46e5-9015-224bd2042f15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8630e75ed37e4ccb989b500c6c44a0fd)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:08.866811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3cfe26-ec34-46e5-9015-224bd2042f15 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:08.930159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ca3cfe26-ec34-46e5-9015-224bd2042f15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8630e75ed37e4ccb989b500c6c44a0fd', 'request_id': 'req-ca3cfe26-ec34-46e5-9015-224bd2042f15'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.942616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ca3cfe26-ec34-46e5-9015-224bd2042f15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8630e75ed37e4ccb989b500c6c44a0fd', 'request_id': 'req-ca3cfe26-ec34-46e5-9015-224bd2042f15'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.950359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ca3cfe26-ec34-46e5-9015-224bd2042f15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8630e75ed37e4ccb989b500c6c44a0fd', 'request_id': 'req-ca3cfe26-ec34-46e5-9015-224bd2042f15'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.965724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ca3cfe26-ec34-46e5-9015-224bd2042f15 admin admin] The token cache is being invalidate because project 8630e75ed37e4ccb989b500c6c44a0fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:08.966115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ca3cfe26-ec34-46e5-9015-224bd2042f15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.966815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ca3cfe26-ec34-46e5-9015-224bd2042f15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:08.968510 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 963/1900] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:08 2022] DELETE /identity/v3/projects/8630e75ed37e4ccb989b500c6c44a0fd => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:39:08.978934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce72152-83f3-4b87-b05a-267435c2ca6b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:08.981316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce72152-83f3-4b87-b05a-267435c2ca6b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:08.981843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce72152-83f3-4b87-b05a-267435c2ca6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.005333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-fce72152-83f3-4b87-b05a-267435c2ca6b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:09.006571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-fce72152-83f3-4b87-b05a-267435c2ca6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:09.077470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 938/1901] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:09.086652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43123a-0935-469d-b82d-5f40d1d2d7aa None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.088273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43123a-0935-469d-b82d-5f40d1d2d7aa None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.088558 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43123a-0935-469d-b82d-5f40d1d2d7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.116700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5f43123a-0935-469d-b82d-5f40d1d2d7aa None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:09.118088 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5f43123a-0935-469d-b82d-5f40d1d2d7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:09.128147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 964/1902] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:09.132668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce72152-83f3-4b87-b05a-267435c2ca6b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:09.143840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce72152-83f3-4b87-b05a-267435c2ca6b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:09.148522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedccc6d-3007-43aa-bc1b-aefee92cdc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:09.149171 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedccc6d-3007-43aa-bc1b-aefee92cdc0d admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.149456 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedccc6d-3007-43aa-bc1b-aefee92cdc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.149733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedccc6d-3007-43aa-bc1b-aefee92cdc0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.150379 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedccc6d-3007-43aa-bc1b-aefee92cdc0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:39:09.150688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedccc6d-3007-43aa-bc1b-aefee92cdc0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:09.152279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedccc6d-3007-43aa-bc1b-aefee92cdc0d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:09.164136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 939/1903] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:39:09.169372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43123a-0935-469d-b82d-5f40d1d2d7aa None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:09.178756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43123a-0935-469d-b82d-5f40d1d2d7aa None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:09.183987 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854eaa2-65ee-4727-a5f1-1997dcfe35ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:09.184719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854eaa2-65ee-4727-a5f1-1997dcfe35ab admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.185007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854eaa2-65ee-4727-a5f1-1997dcfe35ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.185437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854eaa2-65ee-4727-a5f1-1997dcfe35ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.186129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854eaa2-65ee-4727-a5f1-1997dcfe35ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:09.188543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854eaa2-65ee-4727-a5f1-1997dcfe35ab admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:09.219390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 965/1904] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:09 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:09.223608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedccc6d-3007-43aa-bc1b-aefee92cdc0d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:09.233754 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedccc6d-3007-43aa-bc1b-aefee92cdc0d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:09.260210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:09.294342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e79e87-f4e7-4fa9-9470-296bc3e100fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:09.294962 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e79e87-f4e7-4fa9-9470-296bc3e100fb admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.295243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e79e87-f4e7-4fa9-9470-296bc3e100fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.295519 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e79e87-f4e7-4fa9-9470-296bc3e100fb admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.296153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e79e87-f4e7-4fa9-9470-296bc3e100fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:09.298115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e79e87-f4e7-4fa9-9470-296bc3e100fb admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:09.323963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 940/1905] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:09 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:09.331360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854eaa2-65ee-4727-a5f1-1997dcfe35ab admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:09.348928 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854eaa2-65ee-4727-a5f1-1997dcfe35ab admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:09.354800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bf183-4479-4f94-9038-309dc53582e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:09.355443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bf183-4479-4f94-9038-309dc53582e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.355722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bf183-4479-4f94-9038-309dc53582e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.356055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bf183-4479-4f94-9038-309dc53582e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.356661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bf183-4479-4f94-9038-309dc53582e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:09.358381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bf183-4479-4f94-9038-309dc53582e4 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:09.372276 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 966/1906] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:39:09.377654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e79e87-f4e7-4fa9-9470-296bc3e100fb admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:09.391536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e79e87-f4e7-4fa9-9470-296bc3e100fb admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:09.393730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ba51c-2014-46d3-9a5c-a9de878133e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:09.394342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ba51c-2014-46d3-9a5c-a9de878133e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.394629 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ba51c-2014-46d3-9a5c-a9de878133e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.394931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ba51c-2014-46d3-9a5c-a9de878133e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.395582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ba51c-2014-46d3-9a5c-a9de878133e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:09.397218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ba51c-2014-46d3-9a5c-a9de878133e7 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:09.409687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 941/1907] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:09.415154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bf183-4479-4f94-9038-309dc53582e4 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:09.425746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bf183-4479-4f94-9038-309dc53582e4 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:09.430508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d54c63-d131-48aa-8a0c-7b2621078b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:09.431184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d54c63-d131-48aa-8a0c-7b2621078b89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.431466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d54c63-d131-48aa-8a0c-7b2621078b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.431744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d54c63-d131-48aa-8a0c-7b2621078b89 admin admin] PATH_INFO: `/v3/projects/05e677715cdd4ac4b79141c98bddf090/users/48ec95af18ef44ebb5f63bbb26801820/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.448201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d54c63-d131-48aa-8a0c-7b2621078b89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e677715cdd4ac4b79141c98bddf090, user_id=48ec95af18ef44ebb5f63bbb26801820, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:09.450891 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d54c63-d131-48aa-8a0c-7b2621078b89 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:09.467120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 967/1908] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:09 2022] PUT /identity/v3/projects/05e677715cdd4ac4b79141c98bddf090/users/48ec95af18ef44ebb5f63bbb26801820/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:09.478259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f8e8d-b2e2-4391-9af8-0643688e4fa0 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.478697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f8e8d-b2e2-4391-9af8-0643688e4fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.478905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f8e8d-b2e2-4391-9af8-0643688e4fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.508853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-fc4f8e8d-b2e2-4391-9af8-0643688e4fa0 None None] MFA Rules not processed for user `48ec95af18ef44ebb5f63bbb26801820`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:09.510289 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-fc4f8e8d-b2e2-4391-9af8-0643688e4fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:09.598852 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 942/1909] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:09.603434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d54c63-d131-48aa-8a0c-7b2621078b89 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:09.659625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:09.723632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d54c63-d131-48aa-8a0c-7b2621078b89 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:09.728779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae92881-e84e-4f70-b121-4f0edbf0cae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:09.729578 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae92881-e84e-4f70-b121-4f0edbf0cae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.729861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae92881-e84e-4f70-b121-4f0edbf0cae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.730137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae92881-e84e-4f70-b121-4f0edbf0cae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.731077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae92881-e84e-4f70-b121-4f0edbf0cae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:09.733138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae92881-e84e-4f70-b121-4f0edbf0cae9 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:09.767607 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 968/1910] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:09 2022] POST /identity/v3/projects => generated 375 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:39:09.771729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f8e8d-b2e2-4391-9af8-0643688e4fa0 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:09.782517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f8e8d-b2e2-4391-9af8-0643688e4fa0 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:09.842414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9646fce-32a1-487c-b4ba-240dbd4fb2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:09.843583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9646fce-32a1-487c-b4ba-240dbd4fb2bc admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.843886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9646fce-32a1-487c-b4ba-240dbd4fb2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.844168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9646fce-32a1-487c-b4ba-240dbd4fb2bc admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.844895 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9646fce-32a1-487c-b4ba-240dbd4fb2bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:09.847438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9646fce-32a1-487c-b4ba-240dbd4fb2bc admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:09.872466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 943/1911] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:09 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:09.876474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae92881-e84e-4f70-b121-4f0edbf0cae9 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:09.886895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae92881-e84e-4f70-b121-4f0edbf0cae9 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:09.891543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96253e-67a1-4391-8146-9f8fe9054b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:09.892148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96253e-67a1-4391-8146-9f8fe9054b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.892431 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96253e-67a1-4391-8146-9f8fe9054b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.892709 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96253e-67a1-4391-8146-9f8fe9054b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.893386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96253e-67a1-4391-8146-9f8fe9054b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:09.894998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96253e-67a1-4391-8146-9f8fe9054b57 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:09.907384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 969/1912] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:09.912160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9646fce-32a1-487c-b4ba-240dbd4fb2bc admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:09.923279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9646fce-32a1-487c-b4ba-240dbd4fb2bc admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:09.928178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02b316-bf5a-4edf-b466-5e361ec4fd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:09.928785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02b316-bf5a-4edf-b466-5e361ec4fd86 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.929178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02b316-bf5a-4edf-b466-5e361ec4fd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.929391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02b316-bf5a-4edf-b466-5e361ec4fd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.930089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02b316-bf5a-4edf-b466-5e361ec4fd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:09.931718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02b316-bf5a-4edf-b466-5e361ec4fd86 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:09.941850 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 944/1913] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:09.946002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96253e-67a1-4391-8146-9f8fe9054b57 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:09.956068 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96253e-67a1-4391-8146-9f8fe9054b57 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:09.960929 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b497a-8c1b-46ed-9495-a47cf333b464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:09.961646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b497a-8c1b-46ed-9495-a47cf333b464 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:09.961983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b497a-8c1b-46ed-9495-a47cf333b464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:09.962280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b497a-8c1b-46ed-9495-a47cf333b464 admin admin] PATH_INFO: `/v3/projects/ec45cd3242d94e31b8598096ac10f485/users/38c261702d4d4533b14095f122480b87/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:09.979184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b497a-8c1b-46ed-9495-a47cf333b464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec45cd3242d94e31b8598096ac10f485, user_id=38c261702d4d4533b14095f122480b87, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:09.981928 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b497a-8c1b-46ed-9495-a47cf333b464 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:09.996281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 970/1914] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:09 2022] PUT /identity/v3/projects/ec45cd3242d94e31b8598096ac10f485/users/38c261702d4d4533b14095f122480b87/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:10.006787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7be63-3015-4e90-9ee9-a884cc3ab799 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:10.007083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7be63-3015-4e90-9ee9-a884cc3ab799 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:10.007367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7be63-3015-4e90-9ee9-a884cc3ab799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:10.036614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-23b7be63-3015-4e90-9ee9-a884cc3ab799 None None] MFA Rules not processed for user `38c261702d4d4533b14095f122480b87`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:10.037898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-23b7be63-3015-4e90-9ee9-a884cc3ab799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:10.042110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:10.105155 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 945/1915] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:10.110259 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b497a-8c1b-46ed-9495-a47cf333b464 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:10.190573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b497a-8c1b-46ed-9495-a47cf333b464 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:10.195366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f9491-1f24-4774-a0b7-a5bc85011784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:10.195986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f9491-1f24-4774-a0b7-a5bc85011784 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:10.196261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f9491-1f24-4774-a0b7-a5bc85011784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:10.196559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f9491-1f24-4774-a0b7-a5bc85011784 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:10.197230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f9491-1f24-4774-a0b7-a5bc85011784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:10.199122 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f9491-1f24-4774-a0b7-a5bc85011784 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:10.227645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 971/1916] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:10 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:39:10.231957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7be63-3015-4e90-9ee9-a884cc3ab799 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:10.251679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7be63-3015-4e90-9ee9-a884cc3ab799 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:10.315341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d51f4-d65d-451c-b419-2d0014221704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:10.315958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d51f4-d65d-451c-b419-2d0014221704 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:10.316240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d51f4-d65d-451c-b419-2d0014221704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:10.316513 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d51f4-d65d-451c-b419-2d0014221704 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:10.317210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d51f4-d65d-451c-b419-2d0014221704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:10.319174 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d51f4-d65d-451c-b419-2d0014221704 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:10.347822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 946/1917] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:10 2022] POST /identity/v3/users => generated 296 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:10.351860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f9491-1f24-4774-a0b7-a5bc85011784 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:10.363855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f9491-1f24-4774-a0b7-a5bc85011784 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:10.368616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7f911-c9f1-4fc7-8ca9-0c4ae84bc444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:10.369275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7f911-c9f1-4fc7-8ca9-0c4ae84bc444 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:10.369560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7f911-c9f1-4fc7-8ca9-0c4ae84bc444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:10.369838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7f911-c9f1-4fc7-8ca9-0c4ae84bc444 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:10.370491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7f911-c9f1-4fc7-8ca9-0c4ae84bc444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:10.372077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7f911-c9f1-4fc7-8ca9-0c4ae84bc444 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:10.384120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 972/1918] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:10.388422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d51f4-d65d-451c-b419-2d0014221704 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:10.398630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d51f4-d65d-451c-b419-2d0014221704 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:10.403397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00714f0-8337-45e6-bbaf-a098085c5ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:10.404213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00714f0-8337-45e6-bbaf-a098085c5ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:10.404498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00714f0-8337-45e6-bbaf-a098085c5ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:10.404887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00714f0-8337-45e6-bbaf-a098085c5ef6 admin admin] PATH_INFO: `/v3/projects/1d4b0a8bcc914edb93532d48e537a54c/users/0896b5a76c83414383376720a8e1e681/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:10.421279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00714f0-8337-45e6-bbaf-a098085c5ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4b0a8bcc914edb93532d48e537a54c, user_id=0896b5a76c83414383376720a8e1e681, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:10.423972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00714f0-8337-45e6-bbaf-a098085c5ef6 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:10.438976 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 947/1919] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:10 2022] PUT /identity/v3/projects/1d4b0a8bcc914edb93532d48e537a54c/users/0896b5a76c83414383376720a8e1e681/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:10.443101 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7f911-c9f1-4fc7-8ca9-0c4ae84bc444 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:10.453011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7f911-c9f1-4fc7-8ca9-0c4ae84bc444 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:10.500823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:10.511536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e74bb-b4e2-41e5-9601-52c533f9affe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:10.512162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e74bb-b4e2-41e5-9601-52c533f9affe admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:10.512445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e74bb-b4e2-41e5-9601-52c533f9affe admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:10.512719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e74bb-b4e2-41e5-9601-52c533f9affe admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:10.513406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e74bb-b4e2-41e5-9601-52c533f9affe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:10.515033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e74bb-b4e2-41e5-9601-52c533f9affe admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:10.525954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 973/1920] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:10.531619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00714f0-8337-45e6-bbaf-a098085c5ef6 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:10.542925 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00714f0-8337-45e6-bbaf-a098085c5ef6 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:10.548044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f072ebe-2232-46ee-a03c-e956e6081079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:10.548754 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f072ebe-2232-46ee-a03c-e956e6081079 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:10.549066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f072ebe-2232-46ee-a03c-e956e6081079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:10.549368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f072ebe-2232-46ee-a03c-e956e6081079 admin admin] PATH_INFO: `/v3/projects/1d4b0a8bcc914edb93532d48e537a54c/users/0896b5a76c83414383376720a8e1e681/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:10.551257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f072ebe-2232-46ee-a03c-e956e6081079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4b0a8bcc914edb93532d48e537a54c, user_id=0896b5a76c83414383376720a8e1e681, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:10.553917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f072ebe-2232-46ee-a03c-e956e6081079 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:10.565168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 948/1921] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:10 2022] PUT /identity/v3/projects/1d4b0a8bcc914edb93532d48e537a54c/users/0896b5a76c83414383376720a8e1e681/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:10.576593 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8280ea4-c542-433b-81e2-55e264ac5858 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:10.576869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8280ea4-c542-433b-81e2-55e264ac5858 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:10.577171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8280ea4-c542-433b-81e2-55e264ac5858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:10.606200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-c8280ea4-c542-433b-81e2-55e264ac5858 None None] MFA Rules not processed for user `0896b5a76c83414383376720a8e1e681`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:10.607428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c8280ea4-c542-433b-81e2-55e264ac5858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:10.685789 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 974/1922] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:10.697485 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f072ebe-2232-46ee-a03c-e956e6081079 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:10.699249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9f072ebe-2232-46ee-a03c-e956e6081079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:10.768727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f072ebe-2232-46ee-a03c-e956e6081079 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:10.784439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c5cf0-16d0-4819-9cb8-19f99907cbf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:10.785535 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c5cf0-16d0-4819-9cb8-19f99907cbf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:10.785817 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c5cf0-16d0-4819-9cb8-19f99907cbf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:10.786089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c5cf0-16d0-4819-9cb8-19f99907cbf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:10.804812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c5cf0-16d0-4819-9cb8-19f99907cbf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:10.808502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c5cf0-16d0-4819-9cb8-19f99907cbf9 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:10.815580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:10.823452 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 949/1923] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:10.868730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8280ea4-c542-433b-81e2-55e264ac5858 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:10.870378 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c8280ea4-c542-433b-81e2-55e264ac5858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:10.883105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8280ea4-c542-433b-81e2-55e264ac5858 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:10.886792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d31da1-9371-4823-ad78-ba4a30fda35c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:10.887494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d31da1-9371-4823-ad78-ba4a30fda35c service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:10.887879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d31da1-9371-4823-ad78-ba4a30fda35c service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:10.888213 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d31da1-9371-4823-ad78-ba4a30fda35c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:10.953603 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d31da1-9371-4823-ad78-ba4a30fda35c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:10.955341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d31da1-9371-4823-ad78-ba4a30fda35c service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:10.965636 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 975/1924] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:19.217983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c5cf0-16d0-4819-9cb8-19f99907cbf9 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:19.289099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c5cf0-16d0-4819-9cb8-19f99907cbf9 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:19.297846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:19.298973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:19.299485 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:19.300000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] PATH_INFO: `/v3/users/48ec95af18ef44ebb5f63bbb26801820` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:19.304941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ec95af18ef44ebb5f63bbb26801820)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:19.308553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:19.379498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ec95af18ef44ebb5f63bbb26801820'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:19.387122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ec95af18ef44ebb5f63bbb26801820'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:19.388252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ec95af18ef44ebb5f63bbb26801820'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:19.388859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ec95af18ef44ebb5f63bbb26801820'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:19.397555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ec95af18ef44ebb5f63bbb26801820'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:19.416266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 950/1925] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:19 2022] DELETE /identity/v3/users/48ec95af18ef44ebb5f63bbb26801820 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:19.421245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2a0ce-d872-4396-99c5-6bd16883ff47 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:19.421801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2a0ce-d872-4396-99c5-6bd16883ff47 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:19.421801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2a0ce-d872-4396-99c5-6bd16883ff47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:19.443137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-7ae2a0ce-d872-4396-99c5-6bd16883ff47 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:19.444362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7ae2a0ce-d872-4396-99c5-6bd16883ff47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:19.565277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 976/1926] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:19.573111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:19.575704 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:19.663396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4538d13-4b42-47f8-a5bc-c2890e0b9b12 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:19.669192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285a3ad-e8e3-4297-af65-086a98ea3363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:19.669924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285a3ad-e8e3-4297-af65-086a98ea3363 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:19.670267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285a3ad-e8e3-4297-af65-086a98ea3363 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:19.670782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285a3ad-e8e3-4297-af65-086a98ea3363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:19.683516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285a3ad-e8e3-4297-af65-086a98ea3363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:19.685760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285a3ad-e8e3-4297-af65-086a98ea3363 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:19.697850 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 951/1927] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:39:20.131449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2a0ce-d872-4396-99c5-6bd16883ff47 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:20.134820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7ae2a0ce-d872-4396-99c5-6bd16883ff47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:20.176069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2a0ce-d872-4396-99c5-6bd16883ff47 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:20.184357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:20.185688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:20.186460 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:20.187501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] PATH_INFO: `/v3/projects/05e677715cdd4ac4b79141c98bddf090` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:20.206453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05e677715cdd4ac4b79141c98bddf090)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:20.222123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:20.237326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:20.294828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05e677715cdd4ac4b79141c98bddf090', 'request_id': 'req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:20.309995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05e677715cdd4ac4b79141c98bddf090', 'request_id': 'req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:20.318324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05e677715cdd4ac4b79141c98bddf090', 'request_id': 'req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:20.333747 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] The token cache is being invalidate because project 05e677715cdd4ac4b79141c98bddf090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:20.334210 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:20.335624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:20.340907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 977/1928] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:20 2022] DELETE /identity/v3/projects/05e677715cdd4ac4b79141c98bddf090 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:20.341413 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285a3ad-e8e3-4297-af65-086a98ea3363 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:20.358784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285a3ad-e8e3-4297-af65-086a98ea3363 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:20.412421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:20.494197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:20.495132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:20.495563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:20.495985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] PATH_INFO: `/v3/users/38c261702d4d4533b14095f122480b87` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:20.497551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] RBAC: Authorizing `identity:delete_user(user_id=38c261702d4d4533b14095f122480b87)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:20.500494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:20.579256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38c261702d4d4533b14095f122480b87'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:20.588265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38c261702d4d4533b14095f122480b87'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:20.589855 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38c261702d4d4533b14095f122480b87'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:20.590505 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38c261702d4d4533b14095f122480b87'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:20.601419 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38c261702d4d4533b14095f122480b87'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:20.642956 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 952/1929] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:20 2022] DELETE /identity/v3/users/38c261702d4d4533b14095f122480b87 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:20.924033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:20.925503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:20.947208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb52309-f1fb-4ee4-a2c9-ddf65154dcb7 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:21.012869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:21.013553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:21.013921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:21.014116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] PATH_INFO: `/v3/projects/ec45cd3242d94e31b8598096ac10f485` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:21.015254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec45cd3242d94e31b8598096ac10f485)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:21.017208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:21.064930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec45cd3242d94e31b8598096ac10f485', 'request_id': 'req-7f71094f-721e-4da4-8522-78bebd90f3a7'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.071538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec45cd3242d94e31b8598096ac10f485', 'request_id': 'req-7f71094f-721e-4da4-8522-78bebd90f3a7'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.078272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec45cd3242d94e31b8598096ac10f485', 'request_id': 'req-7f71094f-721e-4da4-8522-78bebd90f3a7'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.091949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] The token cache is being invalidate because project ec45cd3242d94e31b8598096ac10f485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:21.092246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.093673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.095205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 978/1930] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:20 2022] DELETE /identity/v3/projects/ec45cd3242d94e31b8598096ac10f485 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:21.099242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:21.100637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:21.118600 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76005d-8371-4028-b57f-06461fa0276d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:21.183467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373b497-357d-47e6-8008-94043ecb28e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:21.184631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373b497-357d-47e6-8008-94043ecb28e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:21.184631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373b497-357d-47e6-8008-94043ecb28e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:21.184631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373b497-357d-47e6-8008-94043ecb28e9 admin admin] PATH_INFO: `/v3/users/0896b5a76c83414383376720a8e1e681` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:21.186115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373b497-357d-47e6-8008-94043ecb28e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0896b5a76c83414383376720a8e1e681)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:21.188008 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373b497-357d-47e6-8008-94043ecb28e9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:21.271103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:21.274835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8373b497-357d-47e6-8008-94043ecb28e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0896b5a76c83414383376720a8e1e681'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.302504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8373b497-357d-47e6-8008-94043ecb28e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0896b5a76c83414383376720a8e1e681'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.303322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8373b497-357d-47e6-8008-94043ecb28e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0896b5a76c83414383376720a8e1e681'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.307790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8373b497-357d-47e6-8008-94043ecb28e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0896b5a76c83414383376720a8e1e681'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.322517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8373b497-357d-47e6-8008-94043ecb28e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0896b5a76c83414383376720a8e1e681'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.344598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 953/1931] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:21 2022] DELETE /identity/v3/users/0896b5a76c83414383376720a8e1e681 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:21.642381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:21.643871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:21.673851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71094f-721e-4da4-8522-78bebd90f3a7 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:21.702143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:21.746874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:21.747577 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:21.747864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:21.748134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd admin admin] PATH_INFO: `/v3/projects/1d4b0a8bcc914edb93532d48e537a54c` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:21.749854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d4b0a8bcc914edb93532d48e537a54c)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:21.751795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:21.806406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4b0a8bcc914edb93532d48e537a54c', 'request_id': 'req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.813544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4b0a8bcc914edb93532d48e537a54c', 'request_id': 'req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.819982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4b0a8bcc914edb93532d48e537a54c', 'request_id': 'req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.834090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd admin admin] The token cache is being invalidate because project 1d4b0a8bcc914edb93532d48e537a54c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:21.834388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.835070 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8bc9a13a-6979-48f2-a840-7e4e4e8a5cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:21.841434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 979/1932] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:21 2022] DELETE /identity/v3/projects/1d4b0a8bcc914edb93532d48e537a54c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:21.843448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598b3a8-8b70-42fc-a433-fd4da1e37610 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:21.843737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598b3a8-8b70-42fc-a433-fd4da1e37610 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:21.844012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598b3a8-8b70-42fc-a433-fd4da1e37610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:21.864598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-1598b3a8-8b70-42fc-a433-fd4da1e37610 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:21.865922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1598b3a8-8b70-42fc-a433-fd4da1e37610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:21.932496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 954/1933] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:21.940871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7853bf6-1b7d-4284-8f75-d6530f58ba78 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:21.941364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7853bf6-1b7d-4284-8f75-d6530f58ba78 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:21.941470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7853bf6-1b7d-4284-8f75-d6530f58ba78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:21.961144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e7853bf6-1b7d-4284-8f75-d6530f58ba78 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:21.962370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e7853bf6-1b7d-4284-8f75-d6530f58ba78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:21.970909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 980/1934] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:21.975208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598b3a8-8b70-42fc-a433-fd4da1e37610 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:21.987247 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598b3a8-8b70-42fc-a433-fd4da1e37610 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:21.993777 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcf192-2ba0-43a4-9160-bb8981e771b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:21.994381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcf192-2ba0-43a4-9160-bb8981e771b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:21.994657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcf192-2ba0-43a4-9160-bb8981e771b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:21.994925 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcf192-2ba0-43a4-9160-bb8981e771b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:21.995568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbcf192-2ba0-43a4-9160-bb8981e771b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:39:21.995872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbcf192-2ba0-43a4-9160-bb8981e771b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:21.998045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbcf192-2ba0-43a4-9160-bb8981e771b9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.010823 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 955/1935] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:39:22.016065 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7853bf6-1b7d-4284-8f75-d6530f58ba78 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:22.025486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7853bf6-1b7d-4284-8f75-d6530f58ba78 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:22.030559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b02671-120c-4c73-9ddf-dcdcc197977b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:22.031149 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b02671-120c-4c73-9ddf-dcdcc197977b admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.031421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b02671-120c-4c73-9ddf-dcdcc197977b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.031685 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b02671-120c-4c73-9ddf-dcdcc197977b admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.032336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b02671-120c-4c73-9ddf-dcdcc197977b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:22.034245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b02671-120c-4c73-9ddf-dcdcc197977b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.061186 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 981/1936] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:22 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:22.065367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcf192-2ba0-43a4-9160-bb8981e771b9 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:22.075468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcf192-2ba0-43a4-9160-bb8981e771b9 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:22.130789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59196f9d-2f20-4659-b916-7ed7b2bc4531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:22.131430 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59196f9d-2f20-4659-b916-7ed7b2bc4531 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.131712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59196f9d-2f20-4659-b916-7ed7b2bc4531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.131988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59196f9d-2f20-4659-b916-7ed7b2bc4531 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.132626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59196f9d-2f20-4659-b916-7ed7b2bc4531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:22.134584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59196f9d-2f20-4659-b916-7ed7b2bc4531 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.160178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 956/1937] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:22 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:22.161506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b02671-120c-4c73-9ddf-dcdcc197977b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:22.173134 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b02671-120c-4c73-9ddf-dcdcc197977b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:22.177997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbe1df-68b8-49a1-8f71-d82268d827fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:22.178622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbe1df-68b8-49a1-8f71-d82268d827fa admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.178898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbe1df-68b8-49a1-8f71-d82268d827fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.179192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbe1df-68b8-49a1-8f71-d82268d827fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.179842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbe1df-68b8-49a1-8f71-d82268d827fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:22.181475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbe1df-68b8-49a1-8f71-d82268d827fa admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.193901 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 982/1938] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:22.198374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59196f9d-2f20-4659-b916-7ed7b2bc4531 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:22.209629 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59196f9d-2f20-4659-b916-7ed7b2bc4531 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:22.214593 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e69f6-df23-4dbc-8282-b0de8e89d555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:22.215136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e69f6-df23-4dbc-8282-b0de8e89d555 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.215424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e69f6-df23-4dbc-8282-b0de8e89d555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.215695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e69f6-df23-4dbc-8282-b0de8e89d555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.216347 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e69f6-df23-4dbc-8282-b0de8e89d555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:22.218676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e69f6-df23-4dbc-8282-b0de8e89d555 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.229833 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 957/1939] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:39:22.234026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbe1df-68b8-49a1-8f71-d82268d827fa admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:22.244102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbe1df-68b8-49a1-8f71-d82268d827fa admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:22.248839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48229f7b-c0e2-4d45-a628-3292f88af500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:22.249574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48229f7b-c0e2-4d45-a628-3292f88af500 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.249855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48229f7b-c0e2-4d45-a628-3292f88af500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.250124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48229f7b-c0e2-4d45-a628-3292f88af500 admin admin] PATH_INFO: `/v3/projects/43eb1fc73f5349bb84532ed9ff10c4f6/users/7f453c8a15fa45f493d86b9d9f674a0f/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.265658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48229f7b-c0e2-4d45-a628-3292f88af500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43eb1fc73f5349bb84532ed9ff10c4f6, user_id=7f453c8a15fa45f493d86b9d9f674a0f, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:22.271989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48229f7b-c0e2-4d45-a628-3292f88af500 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.287844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 983/1940] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:22 2022] PUT /identity/v3/projects/43eb1fc73f5349bb84532ed9ff10c4f6/users/7f453c8a15fa45f493d86b9d9f674a0f/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:39:22.304528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94548568-964d-4f47-95c8-b4de28aa81b6 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.304812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94548568-964d-4f47-95c8-b4de28aa81b6 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.305108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94548568-964d-4f47-95c8-b4de28aa81b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.335086 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-94548568-964d-4f47-95c8-b4de28aa81b6 None None] MFA Rules not processed for user `7f453c8a15fa45f493d86b9d9f674a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:22.336354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-94548568-964d-4f47-95c8-b4de28aa81b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:22.350953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:22.446838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 958/1941] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:22.448385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48229f7b-c0e2-4d45-a628-3292f88af500 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:22.517191 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:22.533311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48229f7b-c0e2-4d45-a628-3292f88af500 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:22.544835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434baa82-9adb-4684-b644-b767012310d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:22.545656 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434baa82-9adb-4684-b644-b767012310d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.546015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434baa82-9adb-4684-b644-b767012310d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.546379 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434baa82-9adb-4684-b644-b767012310d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.547178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434baa82-9adb-4684-b644-b767012310d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:22.549881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434baa82-9adb-4684-b644-b767012310d5 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.589770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 984/1942] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:22 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:22.593823 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94548568-964d-4f47-95c8-b4de28aa81b6 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:22.608094 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94548568-964d-4f47-95c8-b4de28aa81b6 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:22.678557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a996d5a-16d9-47c4-9788-1a838221ca2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:22.679454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a996d5a-16d9-47c4-9788-1a838221ca2f admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.682144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a996d5a-16d9-47c4-9788-1a838221ca2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.682501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a996d5a-16d9-47c4-9788-1a838221ca2f admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.683230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a996d5a-16d9-47c4-9788-1a838221ca2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:22.685339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a996d5a-16d9-47c4-9788-1a838221ca2f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.713615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 959/1943] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:22 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:39:22.717798 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434baa82-9adb-4684-b644-b767012310d5 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:22.734646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434baa82-9adb-4684-b644-b767012310d5 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:22.741638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f5d7f-4dfc-4796-9a0f-6a4ce34550f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:22.742345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f5d7f-4dfc-4796-9a0f-6a4ce34550f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.742648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f5d7f-4dfc-4796-9a0f-6a4ce34550f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.743145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f5d7f-4dfc-4796-9a0f-6a4ce34550f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.744162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f5d7f-4dfc-4796-9a0f-6a4ce34550f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:22.746743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f5d7f-4dfc-4796-9a0f-6a4ce34550f2 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.759988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 985/1944] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:22.764175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a996d5a-16d9-47c4-9788-1a838221ca2f admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:22.777396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a996d5a-16d9-47c4-9788-1a838221ca2f admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:22.782968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac6611-ef3c-4936-9550-40959674badf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:22.783699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac6611-ef3c-4936-9550-40959674badf admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.784042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac6611-ef3c-4936-9550-40959674badf admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.784384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac6611-ef3c-4936-9550-40959674badf admin admin] PATH_INFO: `/v3/projects/8fc6ef08b9e64baca2fa4871fef8f402/users/0411c4325de74afba87cb4545e27e435/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.804425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac6611-ef3c-4936-9550-40959674badf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc6ef08b9e64baca2fa4871fef8f402, user_id=0411c4325de74afba87cb4545e27e435, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:22.807460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac6611-ef3c-4936-9550-40959674badf admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.822021 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f5d7f-4dfc-4796-9a0f-6a4ce34550f2 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:22.823822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-293f5d7f-4dfc-4796-9a0f-6a4ce34550f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:22.827047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 960/1945] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:22 2022] PUT /identity/v3/projects/8fc6ef08b9e64baca2fa4871fef8f402/users/0411c4325de74afba87cb4545e27e435/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:22.831414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac6611-ef3c-4936-9550-40959674badf admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:22.837536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f5d7f-4dfc-4796-9a0f-6a4ce34550f2 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:22.907482 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d41ef8-c38e-4e69-93bc-0490cc422140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:22.908723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d41ef8-c38e-4e69-93bc-0490cc422140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.909007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d41ef8-c38e-4e69-93bc-0490cc422140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.909464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d41ef8-c38e-4e69-93bc-0490cc422140 admin admin] PATH_INFO: `/v3/users/45ff264f831e4750b4bd92e90b10ce8d` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.910650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d41ef8-c38e-4e69-93bc-0490cc422140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45ff264f831e4750b4bd92e90b10ce8d)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:22.911883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac6611-ef3c-4936-9550-40959674badf admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:22.912795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d41ef8-c38e-4e69-93bc-0490cc422140 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.916970 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62e635-752e-4fb8-8952-c741f6283a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:22.917733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62e635-752e-4fb8-8952-c741f6283a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.918106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62e635-752e-4fb8-8952-c741f6283a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.918472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc62e635-752e-4fb8-8952-c741f6283a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.919218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62e635-752e-4fb8-8952-c741f6283a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:22.921186 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc62e635-752e-4fb8-8952-c741f6283a6b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.933143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 961/1946] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:39:22.936771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62e635-752e-4fb8-8952-c741f6283a6b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:22.957253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc62e635-752e-4fb8-8952-c741f6283a6b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:22.963277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d6b7f-f134-4637-aaab-2b4448ca4308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:22.963947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d6b7f-f134-4637-aaab-2b4448ca4308 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.964223 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d6b7f-f134-4637-aaab-2b4448ca4308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.964494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d6b7f-f134-4637-aaab-2b4448ca4308 admin admin] PATH_INFO: `/v3/projects/8fc6ef08b9e64baca2fa4871fef8f402/users/0411c4325de74afba87cb4545e27e435/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:22.966458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d6b7f-f134-4637-aaab-2b4448ca4308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc6ef08b9e64baca2fa4871fef8f402, user_id=0411c4325de74afba87cb4545e27e435, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:22.969229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d6b7f-f134-4637-aaab-2b4448ca4308 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:22.973853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d6d41ef8-c38e-4e69-93bc-0490cc422140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45ff264f831e4750b4bd92e90b10ce8d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:22.981191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 962/1947] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:22 2022] PUT /identity/v3/projects/8fc6ef08b9e64baca2fa4871fef8f402/users/0411c4325de74afba87cb4545e27e435/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:39:22.991358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d6d41ef8-c38e-4e69-93bc-0490cc422140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45ff264f831e4750b4bd92e90b10ce8d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:22.992254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d6d41ef8-c38e-4e69-93bc-0490cc422140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45ff264f831e4750b4bd92e90b10ce8d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:22.992398 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592c635-7c61-4ab8-8633-c320307abe5c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:22.992733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592c635-7c61-4ab8-8633-c320307abe5c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:22.993072 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592c635-7c61-4ab8-8633-c320307abe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:23.003903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:23.004334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d6d41ef8-c38e-4e69-93bc-0490cc422140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45ff264f831e4750b4bd92e90b10ce8d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:23.005849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d6d41ef8-c38e-4e69-93bc-0490cc422140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45ff264f831e4750b4bd92e90b10ce8d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:23.022029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 986/1948] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:22 2022] DELETE /identity/v3/users/45ff264f831e4750b4bd92e90b10ce8d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:39:23.026208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95cb18-f70c-4e6c-921c-66c46ce21698 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:23.026512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95cb18-f70c-4e6c-921c-66c46ce21698 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:23.026818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95cb18-f70c-4e6c-921c-66c46ce21698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:23.033264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-4592c635-7c61-4ab8-8633-c320307abe5c None None] MFA Rules not processed for user `0411c4325de74afba87cb4545e27e435`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:23.036381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4592c635-7c61-4ab8-8633-c320307abe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:23.049022 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-ea95cb18-f70c-4e6c-921c-66c46ce21698 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:23.050264 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ea95cb18-f70c-4e6c-921c-66c46ce21698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:23.105367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:23.136062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 987/1949] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:23.145244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 963/1950] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:23.152313 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95cb18-f70c-4e6c-921c-66c46ce21698 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:23.154217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ea95cb18-f70c-4e6c-921c-66c46ce21698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:23.155911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592c635-7c61-4ab8-8633-c320307abe5c None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:23.157619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4592c635-7c61-4ab8-8633-c320307abe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:23.220543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95cb18-f70c-4e6c-921c-66c46ce21698 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:23.224278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379cc6cf-01b3-4146-9b2a-99451a67a3a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:23.224894 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cc6cf-01b3-4146-9b2a-99451a67a3a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:23.225726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cc6cf-01b3-4146-9b2a-99451a67a3a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:23.225726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cc6cf-01b3-4146-9b2a-99451a67a3a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:23.229959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592c635-7c61-4ab8-8633-c320307abe5c None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:23.235132 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465e41a-7657-4ac3-99f9-b2ecc4142725 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:23.235801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379cc6cf-01b3-4146-9b2a-99451a67a3a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:23.235913 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e41a-7657-4ac3-99f9-b2ecc4142725 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:23.236316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e41a-7657-4ac3-99f9-b2ecc4142725 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:23.236597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e41a-7657-4ac3-99f9-b2ecc4142725 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:23.237498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379cc6cf-01b3-4146-9b2a-99451a67a3a9 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:23.248279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 988/1951] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:39:23.250079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465e41a-7657-4ac3-99f9-b2ecc4142725 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:23.251542 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465e41a-7657-4ac3-99f9-b2ecc4142725 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:23.262281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 964/1952] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:23.582889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379cc6cf-01b3-4146-9b2a-99451a67a3a9 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:23.584341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-379cc6cf-01b3-4146-9b2a-99451a67a3a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:23.605655 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379cc6cf-01b3-4146-9b2a-99451a67a3a9 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:23.622403 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:23.623041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:23.623330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:23.623641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] PATH_INFO: `/v3/projects/6a472511439940f390b7b40a73e9e1d1` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:23.624944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a472511439940f390b7b40a73e9e1d1)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:23.627192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:23.706917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a472511439940f390b7b40a73e9e1d1', 'request_id': 'req-bb9293a0-6ff3-4988-add0-27188510f6c6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:23.713721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a472511439940f390b7b40a73e9e1d1', 'request_id': 'req-bb9293a0-6ff3-4988-add0-27188510f6c6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:23.721052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a472511439940f390b7b40a73e9e1d1', 'request_id': 'req-bb9293a0-6ff3-4988-add0-27188510f6c6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:23.734998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] The token cache is being invalidate because project 6a472511439940f390b7b40a73e9e1d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:23.735224 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:23.735861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:23.737430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 989/1953] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:23 2022] DELETE /identity/v3/projects/6a472511439940f390b7b40a73e9e1d1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:23.741623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465e41a-7657-4ac3-99f9-b2ecc4142725 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:23.754621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465e41a-7657-4ac3-99f9-b2ecc4142725 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:23.821874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c582c-6bfb-4484-9cfe-93a160b3ed37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:23.822641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c582c-6bfb-4484-9cfe-93a160b3ed37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:23.823010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c582c-6bfb-4484-9cfe-93a160b3ed37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:23.823407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c582c-6bfb-4484-9cfe-93a160b3ed37 admin admin] PATH_INFO: `/v3/users/59c1951fab3e4134bb2ab679e2a2b21a` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:23.825219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c582c-6bfb-4484-9cfe-93a160b3ed37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c1951fab3e4134bb2ab679e2a2b21a)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:23.828809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c582c-6bfb-4484-9cfe-93a160b3ed37 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:23.872700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:23.887746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c90c582c-6bfb-4484-9cfe-93a160b3ed37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c1951fab3e4134bb2ab679e2a2b21a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:23.898130 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c90c582c-6bfb-4484-9cfe-93a160b3ed37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c1951fab3e4134bb2ab679e2a2b21a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:23.899099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c90c582c-6bfb-4484-9cfe-93a160b3ed37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c1951fab3e4134bb2ab679e2a2b21a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:23.899747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c90c582c-6bfb-4484-9cfe-93a160b3ed37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c1951fab3e4134bb2ab679e2a2b21a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:23.916416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c90c582c-6bfb-4484-9cfe-93a160b3ed37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c1951fab3e4134bb2ab679e2a2b21a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:23.937981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 965/1954] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:23 2022] DELETE /identity/v3/users/59c1951fab3e4134bb2ab679e2a2b21a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:39:23.995953 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:23.997429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:24.011435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9293a0-6ff3-4988-add0-27188510f6c6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:24.078243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:24.079152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:24.079416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:24.079697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] PATH_INFO: `/v3/users/7f453c8a15fa45f493d86b9d9f674a0f` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:24.080861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f453c8a15fa45f493d86b9d9f674a0f)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:24.082958 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:24.096872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:24.142538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f453c8a15fa45f493d86b9d9f674a0f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:24.160991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f453c8a15fa45f493d86b9d9f674a0f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:24.161785 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f453c8a15fa45f493d86b9d9f674a0f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:24.173312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f453c8a15fa45f493d86b9d9f674a0f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:24.174370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f453c8a15fa45f493d86b9d9f674a0f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:24.196120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 990/1955] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:23 2022] DELETE /identity/v3/users/7f453c8a15fa45f493d86b9d9f674a0f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:24.201172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0fc06-cc70-41f2-9be9-014c9dae79ba None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:24.201409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0fc06-cc70-41f2-9be9-014c9dae79ba None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:24.202284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0fc06-cc70-41f2-9be9-014c9dae79ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:24.224209 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-9de0fc06-cc70-41f2-9be9-014c9dae79ba None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:24.226214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9de0fc06-cc70-41f2-9be9-014c9dae79ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:24.330606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 966/1956] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:24.332102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:24.337238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:24.350489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0fc06-cc70-41f2-9be9-014c9dae79ba None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:24.352802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9de0fc06-cc70-41f2-9be9-014c9dae79ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:24.354245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b08d41-e5ad-453a-8ae5-1a7d0076262c admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:24.360200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a50449-2491-44e3-944b-87081c73915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:24.360874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a50449-2491-44e3-944b-87081c73915e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:24.361232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a50449-2491-44e3-944b-87081c73915e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:24.361972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a50449-2491-44e3-944b-87081c73915e admin admin] PATH_INFO: `/v3/projects/0e6566acd7774a1c8da0cd1f65560b0e` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:24.362809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a50449-2491-44e3-944b-87081c73915e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e6566acd7774a1c8da0cd1f65560b0e)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:24.364845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a50449-2491-44e3-944b-87081c73915e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:24.423258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0fc06-cc70-41f2-9be9-014c9dae79ba None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:24.424489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d3a50449-2491-44e3-944b-87081c73915e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e6566acd7774a1c8da0cd1f65560b0e', 'request_id': 'req-d3a50449-2491-44e3-944b-87081c73915e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:24.430385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d3a50449-2491-44e3-944b-87081c73915e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e6566acd7774a1c8da0cd1f65560b0e', 'request_id': 'req-d3a50449-2491-44e3-944b-87081c73915e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:24.437251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d3a50449-2491-44e3-944b-87081c73915e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e6566acd7774a1c8da0cd1f65560b0e', 'request_id': 'req-d3a50449-2491-44e3-944b-87081c73915e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:24.452637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d3a50449-2491-44e3-944b-87081c73915e admin admin] The token cache is being invalidate because project 0e6566acd7774a1c8da0cd1f65560b0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:24.453059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d3a50449-2491-44e3-944b-87081c73915e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:24.454524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d3a50449-2491-44e3-944b-87081c73915e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:24.456224 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 991/1957] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:24 2022] DELETE /identity/v3/projects/0e6566acd7774a1c8da0cd1f65560b0e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:39:24.461206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6353c-c87a-4165-90fd-05b02a07d99d None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:24.461614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6353c-c87a-4165-90fd-05b02a07d99d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:24.461895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6353c-c87a-4165-90fd-05b02a07d99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:24.474768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e533396-d6a4-40a7-97d5-cd24cabdcf5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:24.475388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e533396-d6a4-40a7-97d5-cd24cabdcf5b service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:24.475668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e533396-d6a4-40a7-97d5-cd24cabdcf5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:24.475941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e533396-d6a4-40a7-97d5-cd24cabdcf5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:24.478003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9e533396-d6a4-40a7-97d5-cd24cabdcf5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:24.482080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-92a6353c-c87a-4165-90fd-05b02a07d99d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:24.483951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-92a6353c-c87a-4165-90fd-05b02a07d99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:24.501640 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:24.557807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e533396-d6a4-40a7-97d5-cd24cabdcf5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:24.559299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e533396-d6a4-40a7-97d5-cd24cabdcf5b service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:24.568190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 992/1958] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:24.569846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 967/1959] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:24.576247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e9fb5-9c48-4067-8b0b-a3f4842bad75 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:24.576536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e9fb5-9c48-4067-8b0b-a3f4842bad75 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:24.576817 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e9fb5-9c48-4067-8b0b-a3f4842bad75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:24.597163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-940e9fb5-9c48-4067-8b0b-a3f4842bad75 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:24.598413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-940e9fb5-9c48-4067-8b0b-a3f4842bad75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:24.606929 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 993/1960] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:24.612774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e533396-d6a4-40a7-97d5-cd24cabdcf5b service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:24.635700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e533396-d6a4-40a7-97d5-cd24cabdcf5b service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:24.641598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bce8e0-0434-444f-843d-12bcd263fc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:24.642196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bce8e0-0434-444f-843d-12bcd263fc17 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:24.642503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bce8e0-0434-444f-843d-12bcd263fc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:24.642778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bce8e0-0434-444f-843d-12bcd263fc17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:24.643591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bce8e0-0434-444f-843d-12bcd263fc17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:39:24.644259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bce8e0-0434-444f-843d-12bcd263fc17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:24.645947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bce8e0-0434-444f-843d-12bcd263fc17 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:24.659589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 968/1961] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:39:24.664369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e9fb5-9c48-4067-8b0b-a3f4842bad75 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:24.676208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e9fb5-9c48-4067-8b0b-a3f4842bad75 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:24.682745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53203ac2-95e8-495f-9d02-66e3bb32b803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:24.683363 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53203ac2-95e8-495f-9d02-66e3bb32b803 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:24.683762 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53203ac2-95e8-495f-9d02-66e3bb32b803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:24.684071 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53203ac2-95e8-495f-9d02-66e3bb32b803 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:24.684876 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53203ac2-95e8-495f-9d02-66e3bb32b803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:24.687670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53203ac2-95e8-495f-9d02-66e3bb32b803 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:24.694152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:24.727177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 994/1962] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:24 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:39:24.728453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bce8e0-0434-444f-843d-12bcd263fc17 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:24.745349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bce8e0-0434-444f-843d-12bcd263fc17 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:24.824635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c74eb-afee-4fec-9fbc-28e5219cf9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:24.825416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c74eb-afee-4fec-9fbc-28e5219cf9ea admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:24.825754 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c74eb-afee-4fec-9fbc-28e5219cf9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:24.826399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c74eb-afee-4fec-9fbc-28e5219cf9ea admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:24.826844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c74eb-afee-4fec-9fbc-28e5219cf9ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:24.828803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c74eb-afee-4fec-9fbc-28e5219cf9ea admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:24.854151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 969/1963] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:24 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:24.859246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53203ac2-95e8-495f-9d02-66e3bb32b803 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:24.870287 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53203ac2-95e8-495f-9d02-66e3bb32b803 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:24.878623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c74eb-afee-4fec-9fbc-28e5219cf9ea admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:24.878756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fb05b-a474-403a-a576-c6850f69f6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:24.879201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fb05b-a474-403a-a576-c6850f69f6ca admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:24.879641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fb05b-a474-403a-a576-c6850f69f6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:24.879983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fb05b-a474-403a-a576-c6850f69f6ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:24.880311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1b6c74eb-afee-4fec-9fbc-28e5219cf9ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:24.880778 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fb05b-a474-403a-a576-c6850f69f6ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:24.882474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fb05b-a474-403a-a576-c6850f69f6ca admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:24.895572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c74eb-afee-4fec-9fbc-28e5219cf9ea admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:24.901651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 995/1964] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:24.902220 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:24.903286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:24.903892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:24.905438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] PATH_INFO: `/v3/projects/43eb1fc73f5349bb84532ed9ff10c4f6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:24.907247 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=43eb1fc73f5349bb84532ed9ff10c4f6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:24.909692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:24.913381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fb05b-a474-403a-a576-c6850f69f6ca admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:24.927072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fb05b-a474-403a-a576-c6850f69f6ca admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:24.933439 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1769ad-876f-4aa6-afd7-3b32e8c312fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:24.934124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1769ad-876f-4aa6-afd7-3b32e8c312fd admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:24.934523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1769ad-876f-4aa6-afd7-3b32e8c312fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:24.937125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1769ad-876f-4aa6-afd7-3b32e8c312fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:24.937125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1769ad-876f-4aa6-afd7-3b32e8c312fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:24.938533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1769ad-876f-4aa6-afd7-3b32e8c312fd admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:24.953135 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 996/1965] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:24.954598 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1769ad-876f-4aa6-afd7-3b32e8c312fd admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:24.969796 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1769ad-876f-4aa6-afd7-3b32e8c312fd admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:24.973471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43eb1fc73f5349bb84532ed9ff10c4f6', 'request_id': 'req-b47be92a-30a1-4f37-beda-01bfa55cdb1e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:24.981548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43eb1fc73f5349bb84532ed9ff10c4f6', 'request_id': 'req-b47be92a-30a1-4f37-beda-01bfa55cdb1e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:24.991647 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43eb1fc73f5349bb84532ed9ff10c4f6', 'request_id': 'req-b47be92a-30a1-4f37-beda-01bfa55cdb1e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:25.005574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] The token cache is being invalidate because project 43eb1fc73f5349bb84532ed9ff10c4f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:25.005983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:25.006991 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:25.009360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 970/1966] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:24 2022] DELETE /identity/v3/projects/43eb1fc73f5349bb84532ed9ff10c4f6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:25.013008 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:25.014718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:25.039829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:25.047107 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98df270-19f3-45d4-9bdc-0a93082fe329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:25.047896 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98df270-19f3-45d4-9bdc-0a93082fe329 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:25.048390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98df270-19f3-45d4-9bdc-0a93082fe329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:25.048750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98df270-19f3-45d4-9bdc-0a93082fe329 admin admin] PATH_INFO: `/v3/projects/34b00239804a45488ff8981c0e108f29/users/e5b8a49a8fc44d59919e268088895bd0/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:25.073018 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98df270-19f3-45d4-9bdc-0a93082fe329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34b00239804a45488ff8981c0e108f29, user_id=e5b8a49a8fc44d59919e268088895bd0, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:25.077071 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98df270-19f3-45d4-9bdc-0a93082fe329 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:25.094592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 997/1967] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:24 2022] PUT /identity/v3/projects/34b00239804a45488ff8981c0e108f29/users/e5b8a49a8fc44d59919e268088895bd0/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:39:25.106704 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4057b46-c3a3-4668-a35e-430f18795bea None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:25.108524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4057b46-c3a3-4668-a35e-430f18795bea None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:25.110073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4057b46-c3a3-4668-a35e-430f18795bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:25.113685 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47be92a-30a1-4f37-beda-01bfa55cdb1e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:25.144388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f4057b46-c3a3-4668-a35e-430f18795bea None None] MFA Rules not processed for user `e5b8a49a8fc44d59919e268088895bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:25.147340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f4057b46-c3a3-4668-a35e-430f18795bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:25.188458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:25.189137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:25.189432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:25.189703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] PATH_INFO: `/v3/users/0411c4325de74afba87cb4545e27e435` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:25.191149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0411c4325de74afba87cb4545e27e435)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:25.193609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:25.249099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 998/1968] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:25.253426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4057b46-c3a3-4668-a35e-430f18795bea None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:25.254812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f4057b46-c3a3-4668-a35e-430f18795bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:25.270609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0411c4325de74afba87cb4545e27e435'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:25.271212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4057b46-c3a3-4668-a35e-430f18795bea None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:25.276297 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4e40f-3f69-4206-8c6c-e8f96084771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:25.276865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4e40f-3f69-4206-8c6c-e8f96084771d admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:25.277253 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4e40f-3f69-4206-8c6c-e8f96084771d admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:25.277591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4e40f-3f69-4206-8c6c-e8f96084771d admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:25.278294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4e40f-3f69-4206-8c6c-e8f96084771d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:25.279819 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0411c4325de74afba87cb4545e27e435'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:25.281042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0411c4325de74afba87cb4545e27e435'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:25.281265 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4e40f-3f69-4206-8c6c-e8f96084771d admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:25.281762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0411c4325de74afba87cb4545e27e435'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:25.296549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0411c4325de74afba87cb4545e27e435'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:25.335415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 971/1969] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:25 2022] DELETE /identity/v3/users/0411c4325de74afba87cb4545e27e435 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:39:25.350986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 999/1970] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:25 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:39:25.354341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:25.356157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:25.433594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:25.461866 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9232f4-23b3-48ab-b59d-7734c0bc95e1 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:25.537223 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983e28c-8446-4999-82de-a0cb21c58678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:25.537916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983e28c-8446-4999-82de-a0cb21c58678 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:25.538198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983e28c-8446-4999-82de-a0cb21c58678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:25.538480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983e28c-8446-4999-82de-a0cb21c58678 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:25.539119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983e28c-8446-4999-82de-a0cb21c58678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:25.541613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983e28c-8446-4999-82de-a0cb21c58678 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:25.572194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 972/1971] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:25 2022] POST /identity/v3/users => generated 295 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:25.573981 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4e40f-3f69-4206-8c6c-e8f96084771d admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:25.589825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4e40f-3f69-4206-8c6c-e8f96084771d admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:25.597433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0170a21-1496-48a7-907e-2e73b7e06c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:25.598055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0170a21-1496-48a7-907e-2e73b7e06c62 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:25.598335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0170a21-1496-48a7-907e-2e73b7e06c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:25.598713 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0170a21-1496-48a7-907e-2e73b7e06c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:25.599260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0170a21-1496-48a7-907e-2e73b7e06c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:25.601661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0170a21-1496-48a7-907e-2e73b7e06c62 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:25.616073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1000/1972] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:25.625690 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983e28c-8446-4999-82de-a0cb21c58678 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:25.636169 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983e28c-8446-4999-82de-a0cb21c58678 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:25.641029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d57fe2-50ac-44a7-b323-c86baf95badd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:25.641792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d57fe2-50ac-44a7-b323-c86baf95badd admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:25.642145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d57fe2-50ac-44a7-b323-c86baf95badd admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:25.642491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d57fe2-50ac-44a7-b323-c86baf95badd admin admin] PATH_INFO: `/v3/projects/c719c0c913704c76a8f8a7ff1f2dae62/users/28132135d7244b5ba71b9c53fbcdadd6/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:25.662300 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d57fe2-50ac-44a7-b323-c86baf95badd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c719c0c913704c76a8f8a7ff1f2dae62, user_id=28132135d7244b5ba71b9c53fbcdadd6, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:25.670134 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d57fe2-50ac-44a7-b323-c86baf95badd admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:25.689256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 973/1973] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:25 2022] PUT /identity/v3/projects/c719c0c913704c76a8f8a7ff1f2dae62/users/28132135d7244b5ba71b9c53fbcdadd6/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:25.693646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0170a21-1496-48a7-907e-2e73b7e06c62 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:25.701062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:25.705359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0170a21-1496-48a7-907e-2e73b7e06c62 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:25.789948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446bcd2-6a3c-48a2-872f-7f3ebabba2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:25.790501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446bcd2-6a3c-48a2-872f-7f3ebabba2df admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:25.790787 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446bcd2-6a3c-48a2-872f-7f3ebabba2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:25.791058 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446bcd2-6a3c-48a2-872f-7f3ebabba2df admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:25.791888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446bcd2-6a3c-48a2-872f-7f3ebabba2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:25.793507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446bcd2-6a3c-48a2-872f-7f3ebabba2df admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:25.814425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1001/1974] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:25.819046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d57fe2-50ac-44a7-b323-c86baf95badd admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:25.832483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d57fe2-50ac-44a7-b323-c86baf95badd admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:25.837394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb2ccd-822f-49cc-b868-e52ed8a1ab69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:25.838165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb2ccd-822f-49cc-b868-e52ed8a1ab69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:25.838579 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb2ccd-822f-49cc-b868-e52ed8a1ab69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:25.838994 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb2ccd-822f-49cc-b868-e52ed8a1ab69 admin admin] PATH_INFO: `/v3/projects/c719c0c913704c76a8f8a7ff1f2dae62/users/28132135d7244b5ba71b9c53fbcdadd6/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:25.841276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb2ccd-822f-49cc-b868-e52ed8a1ab69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c719c0c913704c76a8f8a7ff1f2dae62, user_id=28132135d7244b5ba71b9c53fbcdadd6, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:25.845062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb2ccd-822f-49cc-b868-e52ed8a1ab69 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:25.860454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 974/1975] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:25 2022] PUT /identity/v3/projects/c719c0c913704c76a8f8a7ff1f2dae62/users/28132135d7244b5ba71b9c53fbcdadd6/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:25.873944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb2ccd-822f-49cc-b868-e52ed8a1ab69 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:25.874868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708ecee-5b20-49f8-a6d0-0d19fa34d0c0 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:25.875124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708ecee-5b20-49f8-a6d0-0d19fa34d0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:25.875437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708ecee-5b20-49f8-a6d0-0d19fa34d0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:25.876521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3deb2ccd-822f-49cc-b868-e52ed8a1ab69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:25.922752 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e708ecee-5b20-49f8-a6d0-0d19fa34d0c0 None None] MFA Rules not processed for user `28132135d7244b5ba71b9c53fbcdadd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:25.924579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e708ecee-5b20-49f8-a6d0-0d19fa34d0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:25.976292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb2ccd-822f-49cc-b868-e52ed8a1ab69 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:25.983991 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985a207-00a3-48c1-8517-a96d0d668e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:25.985022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985a207-00a3-48c1-8517-a96d0d668e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:25.985727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985a207-00a3-48c1-8517-a96d0d668e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:25.986159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985a207-00a3-48c1-8517-a96d0d668e73 admin admin] PATH_INFO: `/v3/projects/8fc6ef08b9e64baca2fa4871fef8f402` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:25.988076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985a207-00a3-48c1-8517-a96d0d668e73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fc6ef08b9e64baca2fa4871fef8f402)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:25.991746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985a207-00a3-48c1-8517-a96d0d668e73 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:26.018224 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1002/1976] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:26.032392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708ecee-5b20-49f8-a6d0-0d19fa34d0c0 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:26.034178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e708ecee-5b20-49f8-a6d0-0d19fa34d0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:26.064846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:26.075864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-a985a207-00a3-48c1-8517-a96d0d668e73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fc6ef08b9e64baca2fa4871fef8f402', 'request_id': 'req-a985a207-00a3-48c1-8517-a96d0d668e73'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:26.084525 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-a985a207-00a3-48c1-8517-a96d0d668e73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fc6ef08b9e64baca2fa4871fef8f402', 'request_id': 'req-a985a207-00a3-48c1-8517-a96d0d668e73'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:26.098144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-a985a207-00a3-48c1-8517-a96d0d668e73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fc6ef08b9e64baca2fa4871fef8f402', 'request_id': 'req-a985a207-00a3-48c1-8517-a96d0d668e73'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:26.126838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-a985a207-00a3-48c1-8517-a96d0d668e73 admin admin] The token cache is being invalidate because project 8fc6ef08b9e64baca2fa4871fef8f402 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:26.127165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-a985a207-00a3-48c1-8517-a96d0d668e73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:26.131492 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-a985a207-00a3-48c1-8517-a96d0d668e73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:26.133455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 975/1977] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:25 2022] DELETE /identity/v3/projects/8fc6ef08b9e64baca2fa4871fef8f402 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:26.138245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ba5f-c229-418b-a3ae-635fb23cecc7 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:26.138531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ba5f-c229-418b-a3ae-635fb23cecc7 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:26.138809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8ba5f-c229-418b-a3ae-635fb23cecc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:26.141772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708ecee-5b20-49f8-a6d0-0d19fa34d0c0 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:26.179910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-94c8ba5f-c229-418b-a3ae-635fb23cecc7 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:26.181474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-94c8ba5f-c229-418b-a3ae-635fb23cecc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:26.210052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:26.227990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a59bbf-6ad6-4bb9-a7e9-a627c419bda4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:26.228648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a59bbf-6ad6-4bb9-a7e9-a627c419bda4 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:26.229268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a59bbf-6ad6-4bb9-a7e9-a627c419bda4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:26.229767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a59bbf-6ad6-4bb9-a7e9-a627c419bda4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:26.232608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f3a59bbf-6ad6-4bb9-a7e9-a627c419bda4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:26.271249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 976/1978] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:26.279110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32d8b1-30a2-409c-b9f2-9363f05dc775 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:26.279670 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32d8b1-30a2-409c-b9f2-9363f05dc775 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:26.280619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32d8b1-30a2-409c-b9f2-9363f05dc775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:26.322048 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-1d32d8b1-30a2-409c-b9f2-9363f05dc775 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:26.323890 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1d32d8b1-30a2-409c-b9f2-9363f05dc775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:26.339468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 977/1979] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:26.340477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32d8b1-30a2-409c-b9f2-9363f05dc775 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:26.343569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a59bbf-6ad6-4bb9-a7e9-a627c419bda4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:26.346125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a59bbf-6ad6-4bb9-a7e9-a627c419bda4 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:26.351549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32d8b1-30a2-409c-b9f2-9363f05dc775 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:26.358330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da30fb-01a5-4e41-88a9-8e9faa282392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:26.358972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da30fb-01a5-4e41-88a9-8e9faa282392 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:26.359259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da30fb-01a5-4e41-88a9-8e9faa282392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:26.359672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da30fb-01a5-4e41-88a9-8e9faa282392 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:26.360125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1003/1980] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:26.369605 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da30fb-01a5-4e41-88a9-8e9faa282392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:39:26.370942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da30fb-01a5-4e41-88a9-8e9faa282392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:26.374702 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da30fb-01a5-4e41-88a9-8e9faa282392 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:26.395269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 978/1981] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:39:26.396979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a59bbf-6ad6-4bb9-a7e9-a627c419bda4 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:26.408852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a59bbf-6ad6-4bb9-a7e9-a627c419bda4 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:26.415182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cabb8-36a3-4cc2-9be3-852c05f87251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:26.415800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cabb8-36a3-4cc2-9be3-852c05f87251 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:26.416157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cabb8-36a3-4cc2-9be3-852c05f87251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:26.416448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cabb8-36a3-4cc2-9be3-852c05f87251 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:26.417829 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727cabb8-36a3-4cc2-9be3-852c05f87251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:26.419808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727cabb8-36a3-4cc2-9be3-852c05f87251 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:26.447195 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da30fb-01a5-4e41-88a9-8e9faa282392 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:26.449801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d8da30fb-01a5-4e41-88a9-8e9faa282392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:26.459238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1004/1982] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:26 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:39:26.461045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cabb8-36a3-4cc2-9be3-852c05f87251 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:26.464207 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da30fb-01a5-4e41-88a9-8e9faa282392 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:26.528421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727cabb8-36a3-4cc2-9be3-852c05f87251 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:26.529475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bba932-3de0-4a48-b482-2cee84c91f12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:26.530292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bba932-3de0-4a48-b482-2cee84c91f12 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:26.530697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bba932-3de0-4a48-b482-2cee84c91f12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:26.531100 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bba932-3de0-4a48-b482-2cee84c91f12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:26.533158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e13bf7-0732-4417-b25d-8d491b077f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:26.533340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-96bba932-3de0-4a48-b482-2cee84c91f12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:26.533698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e13bf7-0732-4417-b25d-8d491b077f16 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:26.533975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e13bf7-0732-4417-b25d-8d491b077f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:26.534250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e13bf7-0732-4417-b25d-8d491b077f16 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:26.534886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e13bf7-0732-4417-b25d-8d491b077f16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:26.536793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e13bf7-0732-4417-b25d-8d491b077f16 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:26.559105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1005/1983] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:26 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:26.563817 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e13bf7-0732-4417-b25d-8d491b077f16 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:26.574911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e13bf7-0732-4417-b25d-8d491b077f16 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:26.580231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191955b-f8ab-4d41-add9-d1317049f00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:26.580839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191955b-f8ab-4d41-add9-d1317049f00f admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:26.581149 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191955b-f8ab-4d41-add9-d1317049f00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:26.581450 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191955b-f8ab-4d41-add9-d1317049f00f admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:26.582098 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191955b-f8ab-4d41-add9-d1317049f00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:26.583724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191955b-f8ab-4d41-add9-d1317049f00f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:26.594670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1006/1984] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:26.596425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bba932-3de0-4a48-b482-2cee84c91f12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:26.598079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bba932-3de0-4a48-b482-2cee84c91f12 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:26.599928 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191955b-f8ab-4d41-add9-d1317049f00f admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:26.608965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 979/1985] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:39:26.610055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191955b-f8ab-4d41-add9-d1317049f00f admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:26.614691 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11cc7c-55d3-41c1-9ac8-4898b1241d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:26.615266 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11cc7c-55d3-41c1-9ac8-4898b1241d59 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:26.615545 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11cc7c-55d3-41c1-9ac8-4898b1241d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:26.615820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11cc7c-55d3-41c1-9ac8-4898b1241d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:26.616461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11cc7c-55d3-41c1-9ac8-4898b1241d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:26.618062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11cc7c-55d3-41c1-9ac8-4898b1241d59 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:26.628995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1007/1986] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:39:26.632995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bba932-3de0-4a48-b482-2cee84c91f12 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:26.651073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bba932-3de0-4a48-b482-2cee84c91f12 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:26.658416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dee7e8-1860-4813-bddf-eb728ab506d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:26.659275 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dee7e8-1860-4813-bddf-eb728ab506d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:26.659608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dee7e8-1860-4813-bddf-eb728ab506d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:26.660040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dee7e8-1860-4813-bddf-eb728ab506d9 admin admin] PATH_INFO: `/v3/projects/09c98239d7f3486ebc429aad02391120/users/bc811a83ad8c4964a0b92c7091fe9f94/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:26.666572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:26.683838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dee7e8-1860-4813-bddf-eb728ab506d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c98239d7f3486ebc429aad02391120, user_id=bc811a83ad8c4964a0b92c7091fe9f94, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:26.689560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dee7e8-1860-4813-bddf-eb728ab506d9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:26.706965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 980/1987] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:26 2022] PUT /identity/v3/projects/09c98239d7f3486ebc429aad02391120/users/bc811a83ad8c4964a0b92c7091fe9f94/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:26.719917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bc6d3-303e-44b8-9e4f-d0642a8cb2ba None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:26.720408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bc6d3-303e-44b8-9e4f-d0642a8cb2ba None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:26.720975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bc6d3-303e-44b8-9e4f-d0642a8cb2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:26.757125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-d15bc6d3-303e-44b8-9e4f-d0642a8cb2ba None None] MFA Rules not processed for user `bc811a83ad8c4964a0b92c7091fe9f94`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:26.758432 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d15bc6d3-303e-44b8-9e4f-d0642a8cb2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:26.791584 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:26.839375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1008/1988] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:26.844003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dee7e8-1860-4813-bddf-eb728ab506d9 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:26.919838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dee7e8-1860-4813-bddf-eb728ab506d9 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:26.924903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e168fc3-5367-421f-831d-2f401c632b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:26.925540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e168fc3-5367-421f-831d-2f401c632b8e admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:26.925822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e168fc3-5367-421f-831d-2f401c632b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:26.926092 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e168fc3-5367-421f-831d-2f401c632b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:26.926713 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e168fc3-5367-421f-831d-2f401c632b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:26.928612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e168fc3-5367-421f-831d-2f401c632b8e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:26.955139 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 981/1989] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:26 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:26.959193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bc6d3-303e-44b8-9e4f-d0642a8cb2ba None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:26.969744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bc6d3-303e-44b8-9e4f-d0642a8cb2ba None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:27.029782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5875e3f-6938-402d-847e-8ace32793633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:27.030385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5875e3f-6938-402d-847e-8ace32793633 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:27.030665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5875e3f-6938-402d-847e-8ace32793633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:27.030991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5875e3f-6938-402d-847e-8ace32793633 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:27.031560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5875e3f-6938-402d-847e-8ace32793633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:27.033534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5875e3f-6938-402d-847e-8ace32793633 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:27.056059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1009/1990] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:26 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:27.060023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e168fc3-5367-421f-831d-2f401c632b8e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:27.070972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e168fc3-5367-421f-831d-2f401c632b8e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:27.075493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a15616-8e09-43a3-aae8-45d799397b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:27.076094 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a15616-8e09-43a3-aae8-45d799397b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:27.076371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a15616-8e09-43a3-aae8-45d799397b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:27.076643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a15616-8e09-43a3-aae8-45d799397b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:27.077311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a15616-8e09-43a3-aae8-45d799397b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:27.078926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a15616-8e09-43a3-aae8-45d799397b35 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:27.089319 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 982/1991] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:27.093765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5875e3f-6938-402d-847e-8ace32793633 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:27.103801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5875e3f-6938-402d-847e-8ace32793633 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:27.108923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bba31c-1498-4f7f-aad0-3c925c2b4b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:27.109547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bba31c-1498-4f7f-aad0-3c925c2b4b24 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:27.109824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bba31c-1498-4f7f-aad0-3c925c2b4b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:27.110091 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bba31c-1498-4f7f-aad0-3c925c2b4b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:27.110751 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bba31c-1498-4f7f-aad0-3c925c2b4b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:27.112312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bba31c-1498-4f7f-aad0-3c925c2b4b24 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:27.122246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1010/1992] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:27.126240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a15616-8e09-43a3-aae8-45d799397b35 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:27.137214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a15616-8e09-43a3-aae8-45d799397b35 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:27.142442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fc56c-4c2e-4630-bc6c-7ff049df5a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:27.143040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fc56c-4c2e-4630-bc6c-7ff049df5a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:27.143328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fc56c-4c2e-4630-bc6c-7ff049df5a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:27.143612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fc56c-4c2e-4630-bc6c-7ff049df5a2e admin admin] PATH_INFO: `/v3/projects/aa45278fe2c4439c9b1015a34c41405e/users/ee44fccf81784e78afcbccdb4d07223a/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:27.160596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fc56c-4c2e-4630-bc6c-7ff049df5a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa45278fe2c4439c9b1015a34c41405e, user_id=ee44fccf81784e78afcbccdb4d07223a, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:27.163285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fc56c-4c2e-4630-bc6c-7ff049df5a2e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:27.177706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 983/1993] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:27 2022] PUT /identity/v3/projects/aa45278fe2c4439c9b1015a34c41405e/users/ee44fccf81784e78afcbccdb4d07223a/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:27.188575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9fbed-de06-470d-be72-56ea60fd701c None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:27.189291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9fbed-de06-470d-be72-56ea60fd701c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:27.189291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9fbed-de06-470d-be72-56ea60fd701c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:27.226194 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b0b9fbed-de06-470d-be72-56ea60fd701c None None] MFA Rules not processed for user `ee44fccf81784e78afcbccdb4d07223a`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:27.227424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b0b9fbed-de06-470d-be72-56ea60fd701c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:27.309658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1011/1994] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:27.314163 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fc56c-4c2e-4630-bc6c-7ff049df5a2e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:27.392609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fc56c-4c2e-4630-bc6c-7ff049df5a2e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:27.398039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf1d45-1276-47f5-998c-27d3d19d1c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:27.398656 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf1d45-1276-47f5-998c-27d3d19d1c62 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:27.398939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf1d45-1276-47f5-998c-27d3d19d1c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:27.399212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf1d45-1276-47f5-998c-27d3d19d1c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:27.399854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf1d45-1276-47f5-998c-27d3d19d1c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:27.401802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf1d45-1276-47f5-998c-27d3d19d1c62 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:27.430373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 984/1995] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:27 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:27.434474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9fbed-de06-470d-be72-56ea60fd701c None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:27.445405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9fbed-de06-470d-be72-56ea60fd701c None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:27.481216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:27.537748 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916ff0b-f04e-4304-8e27-fa96186b218d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:27.538257 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916ff0b-f04e-4304-8e27-fa96186b218d admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:27.538537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916ff0b-f04e-4304-8e27-fa96186b218d admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:27.538810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916ff0b-f04e-4304-8e27-fa96186b218d admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:27.539540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916ff0b-f04e-4304-8e27-fa96186b218d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:27.541888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916ff0b-f04e-4304-8e27-fa96186b218d admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:27.566166 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1012/1996] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:27 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:27.570048 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf1d45-1276-47f5-998c-27d3d19d1c62 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:27.582687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf1d45-1276-47f5-998c-27d3d19d1c62 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:27.587786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4f615-2c32-4330-8c18-e64f412f50ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:27.588522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4f615-2c32-4330-8c18-e64f412f50ad admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:27.588795 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4f615-2c32-4330-8c18-e64f412f50ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:27.589067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4f615-2c32-4330-8c18-e64f412f50ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:27.589758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4f615-2c32-4330-8c18-e64f412f50ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:27.591389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4f615-2c32-4330-8c18-e64f412f50ad admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:27.606792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 985/1997] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:27.608454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916ff0b-f04e-4304-8e27-fa96186b218d admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:27.622060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916ff0b-f04e-4304-8e27-fa96186b218d admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:27.627086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da0698-94fd-4d07-aff1-1fe8a505b1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:27.628339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da0698-94fd-4d07-aff1-1fe8a505b1ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:27.628630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da0698-94fd-4d07-aff1-1fe8a505b1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:27.628912 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da0698-94fd-4d07-aff1-1fe8a505b1ba admin admin] PATH_INFO: `/v3/projects/0b565c2cc25b4faf898dc29de7c5d908/users/a0ef95ca2e55488f83a13202382aa04c/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:27.683472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da0698-94fd-4d07-aff1-1fe8a505b1ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b565c2cc25b4faf898dc29de7c5d908, user_id=a0ef95ca2e55488f83a13202382aa04c, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:27.688257 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da0698-94fd-4d07-aff1-1fe8a505b1ba admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:27.709130 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4f615-2c32-4330-8c18-e64f412f50ad admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:27.709580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1013/1998] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:27 2022] PUT /identity/v3/projects/0b565c2cc25b4faf898dc29de7c5d908/users/a0ef95ca2e55488f83a13202382aa04c/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:27.721300 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4f615-2c32-4330-8c18-e64f412f50ad admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:27.728181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:27.796047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ce70e-cdbe-41d0-9b66-10540c7c357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:27.796774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ce70e-cdbe-41d0-9b66-10540c7c357f admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:27.797116 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ce70e-cdbe-41d0-9b66-10540c7c357f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:27.797528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ce70e-cdbe-41d0-9b66-10540c7c357f admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:27.798303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ce70e-cdbe-41d0-9b66-10540c7c357f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:27.799961 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ce70e-cdbe-41d0-9b66-10540c7c357f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:27.812536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 986/1999] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:27.816514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da0698-94fd-4d07-aff1-1fe8a505b1ba admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:27.827634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da0698-94fd-4d07-aff1-1fe8a505b1ba admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:27.833838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91d96d-8032-47c4-8ca7-a81bbbb5fa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:27.834648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91d96d-8032-47c4-8ca7-a81bbbb5fa8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:27.835037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91d96d-8032-47c4-8ca7-a81bbbb5fa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:27.835421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91d96d-8032-47c4-8ca7-a81bbbb5fa8b admin admin] PATH_INFO: `/v3/projects/0b565c2cc25b4faf898dc29de7c5d908/users/a0ef95ca2e55488f83a13202382aa04c/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:27.837670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91d96d-8032-47c4-8ca7-a81bbbb5fa8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b565c2cc25b4faf898dc29de7c5d908, user_id=a0ef95ca2e55488f83a13202382aa04c, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:27.840640 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91d96d-8032-47c4-8ca7-a81bbbb5fa8b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:27.853393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1014/2000] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:27 2022] PUT /identity/v3/projects/0b565c2cc25b4faf898dc29de7c5d908/users/a0ef95ca2e55488f83a13202382aa04c/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:27.864362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c859d-ceb2-4561-9ae8-48619c3ff52c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:27.864839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c859d-ceb2-4561-9ae8-48619c3ff52c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:27.864930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c859d-ceb2-4561-9ae8-48619c3ff52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:27.930157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-0d9c859d-ceb2-4561-9ae8-48619c3ff52c None None] MFA Rules not processed for user `a0ef95ca2e55488f83a13202382aa04c`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:27.931469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-0d9c859d-ceb2-4561-9ae8-48619c3ff52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:28.035974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 987/2001] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:28.049366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91d96d-8032-47c4-8ca7-a81bbbb5fa8b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:28.051423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-6e91d96d-8032-47c4-8ca7-a81bbbb5fa8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:28.120044 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91d96d-8032-47c4-8ca7-a81bbbb5fa8b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:28.125227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b66824-0ec3-45ab-9962-c28ff57d765b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:28.126142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b66824-0ec3-45ab-9962-c28ff57d765b service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:28.126626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b66824-0ec3-45ab-9962-c28ff57d765b service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:28.127118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b66824-0ec3-45ab-9962-c28ff57d765b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:28.206461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b66824-0ec3-45ab-9962-c28ff57d765b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:28.208361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b66824-0ec3-45ab-9962-c28ff57d765b service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:28.219578 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1015/2002] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:31.880387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c859d-ceb2-4561-9ae8-48619c3ff52c None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:31.890200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c859d-ceb2-4561-9ae8-48619c3ff52c None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:31.893752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee2846-371c-4f92-832f-92ad119aae95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:31.894357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee2846-371c-4f92-832f-92ad119aae95 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:31.894635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee2846-371c-4f92-832f-92ad119aae95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:31.894906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee2846-371c-4f92-832f-92ad119aae95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:31.953057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee2846-371c-4f92-832f-92ad119aae95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:31.954548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee2846-371c-4f92-832f-92ad119aae95 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:31.963630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 988/2003] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:33.710064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b66824-0ec3-45ab-9962-c28ff57d765b service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:33.720010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b66824-0ec3-45ab-9962-c28ff57d765b service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:33.723719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76016a-cf9a-4d07-a7c6-37de708b16d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:33.724375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76016a-cf9a-4d07-a7c6-37de708b16d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:33.724650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76016a-cf9a-4d07-a7c6-37de708b16d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:33.725037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76016a-cf9a-4d07-a7c6-37de708b16d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:33.734185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76016a-cf9a-4d07-a7c6-37de708b16d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:33.735622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76016a-cf9a-4d07-a7c6-37de708b16d7 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:33.745487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1016/2004] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:41.280281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee2846-371c-4f92-832f-92ad119aae95 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:41.282238 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9fee2846-371c-4f92-832f-92ad119aae95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:41.317879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:41.352530 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee2846-371c-4f92-832f-92ad119aae95 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:41.357122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0004a4-f32d-4140-b485-ba9be74bdb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:41.357898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0004a4-f32d-4140-b485-ba9be74bdb19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:41.358175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0004a4-f32d-4140-b485-ba9be74bdb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:41.358447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0004a4-f32d-4140-b485-ba9be74bdb19 admin admin] PATH_INFO: `/v3/users/a32aa66a7b7746f998aec78df1b5cbef` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:41.359581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0004a4-f32d-4140-b485-ba9be74bdb19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a32aa66a7b7746f998aec78df1b5cbef)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:41.362221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0004a4-f32d-4140-b485-ba9be74bdb19 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:41.414332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0e0004a4-f32d-4140-b485-ba9be74bdb19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a32aa66a7b7746f998aec78df1b5cbef'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:41.421594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0e0004a4-f32d-4140-b485-ba9be74bdb19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a32aa66a7b7746f998aec78df1b5cbef'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:41.422880 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0e0004a4-f32d-4140-b485-ba9be74bdb19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a32aa66a7b7746f998aec78df1b5cbef'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:41.423499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0e0004a4-f32d-4140-b485-ba9be74bdb19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a32aa66a7b7746f998aec78df1b5cbef'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:41.432446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0e0004a4-f32d-4140-b485-ba9be74bdb19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a32aa66a7b7746f998aec78df1b5cbef'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:41.447617 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76016a-cf9a-4d07-a7c6-37de708b16d7 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:41.453685 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 989/2005] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:41 2022] DELETE /identity/v3/users/a32aa66a7b7746f998aec78df1b5cbef => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:41.458007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c18eb-dae3-4a9a-a12d-62d39ebcddcf None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:41.458183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76016a-cf9a-4d07-a7c6-37de708b16d7 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:41.458282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c18eb-dae3-4a9a-a12d-62d39ebcddcf None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:41.458561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c18eb-dae3-4a9a-a12d-62d39ebcddcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:41.481297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-637c18eb-dae3-4a9a-a12d-62d39ebcddcf None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:41.483009 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-637c18eb-dae3-4a9a-a12d-62d39ebcddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:41.488780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:41.528779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e031a7-c1e6-485c-8d4d-19715853dca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:41.529528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e031a7-c1e6-485c-8d4d-19715853dca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:41.529812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e031a7-c1e6-485c-8d4d-19715853dca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:41.530091 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e031a7-c1e6-485c-8d4d-19715853dca2 admin admin] PATH_INFO: `/v3/users/bc811a83ad8c4964a0b92c7091fe9f94` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:41.531344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e031a7-c1e6-485c-8d4d-19715853dca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc811a83ad8c4964a0b92c7091fe9f94)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:41.533772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e031a7-c1e6-485c-8d4d-19715853dca2 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:41.546464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 990/2006] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:41.558072 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c18eb-dae3-4a9a-a12d-62d39ebcddcf None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:41.560002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-637c18eb-dae3-4a9a-a12d-62d39ebcddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:41.586157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-47e031a7-c1e6-485c-8d4d-19715853dca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc811a83ad8c4964a0b92c7091fe9f94'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:41.603242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-47e031a7-c1e6-485c-8d4d-19715853dca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc811a83ad8c4964a0b92c7091fe9f94'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:41.604375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-47e031a7-c1e6-485c-8d4d-19715853dca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc811a83ad8c4964a0b92c7091fe9f94'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:41.612097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-47e031a7-c1e6-485c-8d4d-19715853dca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc811a83ad8c4964a0b92c7091fe9f94'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:41.612716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-47e031a7-c1e6-485c-8d4d-19715853dca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc811a83ad8c4964a0b92c7091fe9f94'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:41.619448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:41.624218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c18eb-dae3-4a9a-a12d-62d39ebcddcf None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:41.628915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1017/2007] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:41 2022] DELETE /identity/v3/users/bc811a83ad8c4964a0b92c7091fe9f94 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:39:41.633727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbdd7a-669f-448b-a180-9320855d0d1a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:41.634019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbdd7a-669f-448b-a180-9320855d0d1a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:41.634304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbdd7a-669f-448b-a180-9320855d0d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:41.635556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c21ec6-32aa-4acc-98b9-ce8cf2da6b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:41.650062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c21ec6-32aa-4acc-98b9-ce8cf2da6b7d service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:41.650062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c21ec6-32aa-4acc-98b9-ce8cf2da6b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:41.650062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c21ec6-32aa-4acc-98b9-ce8cf2da6b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:41.650062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d3c21ec6-32aa-4acc-98b9-ce8cf2da6b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:41.654149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c21ec6-32aa-4acc-98b9-ce8cf2da6b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:41.655289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b9bbdd7a-669f-448b-a180-9320855d0d1a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:41.655610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c21ec6-32aa-4acc-98b9-ce8cf2da6b7d service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:41.656554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b9bbdd7a-669f-448b-a180-9320855d0d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:41.667103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 991/2008] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:39:41.793482 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1018/2009] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:41.812378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c21ec6-32aa-4acc-98b9-ce8cf2da6b7d service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:41.813922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d3c21ec6-32aa-4acc-98b9-ce8cf2da6b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:41.825995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c21ec6-32aa-4acc-98b9-ce8cf2da6b7d service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:41.883552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88dbf8d-aa86-4135-b820-fbd8e10c16ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:41.884279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dbf8d-aa86-4135-b820-fbd8e10c16ba service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:41.884630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dbf8d-aa86-4135-b820-fbd8e10c16ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:41.884977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dbf8d-aa86-4135-b820-fbd8e10c16ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:41.895540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88dbf8d-aa86-4135-b820-fbd8e10c16ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:41.896985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88dbf8d-aa86-4135-b820-fbd8e10c16ba service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:41.909125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 992/2010] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:42.127394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbdd7a-669f-448b-a180-9320855d0d1a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:42.129025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b9bbdd7a-669f-448b-a180-9320855d0d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:42.148042 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbdd7a-669f-448b-a180-9320855d0d1a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:42.153057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04b05d-984a-4f2a-9d29-a5cb88703b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:42.153814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04b05d-984a-4f2a-9d29-a5cb88703b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:42.154346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04b05d-984a-4f2a-9d29-a5cb88703b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:42.154866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04b05d-984a-4f2a-9d29-a5cb88703b57 admin admin] PATH_INFO: `/v3/projects/a40205da07f242a48c7ee37350a1d3bf` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:42.156444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04b05d-984a-4f2a-9d29-a5cb88703b57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a40205da07f242a48c7ee37350a1d3bf)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:42.158975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04b05d-984a-4f2a-9d29-a5cb88703b57 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:42.209572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:42.219320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bf04b05d-984a-4f2a-9d29-a5cb88703b57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a40205da07f242a48c7ee37350a1d3bf', 'request_id': 'req-bf04b05d-984a-4f2a-9d29-a5cb88703b57'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.233629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bf04b05d-984a-4f2a-9d29-a5cb88703b57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a40205da07f242a48c7ee37350a1d3bf', 'request_id': 'req-bf04b05d-984a-4f2a-9d29-a5cb88703b57'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.240815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bf04b05d-984a-4f2a-9d29-a5cb88703b57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a40205da07f242a48c7ee37350a1d3bf', 'request_id': 'req-bf04b05d-984a-4f2a-9d29-a5cb88703b57'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.255482 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bf04b05d-984a-4f2a-9d29-a5cb88703b57 admin admin] The token cache is being invalidate because project a40205da07f242a48c7ee37350a1d3bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:42.255787 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bf04b05d-984a-4f2a-9d29-a5cb88703b57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.256498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bf04b05d-984a-4f2a-9d29-a5cb88703b57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.261537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1019/2011] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:42 2022] DELETE /identity/v3/projects/a40205da07f242a48c7ee37350a1d3bf => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:42.263714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ebb79-6ad4-4e7f-a393-3cbd2932f2cd None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:42.264570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ebb79-6ad4-4e7f-a393-3cbd2932f2cd None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:42.265267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ebb79-6ad4-4e7f-a393-3cbd2932f2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:42.293529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-816ebb79-6ad4-4e7f-a393-3cbd2932f2cd None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:42.294978 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-816ebb79-6ad4-4e7f-a393-3cbd2932f2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:42.369888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 993/2012] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:42.378869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138307e8-b15a-4f59-a787-18f0dc902e50 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:42.379149 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138307e8-b15a-4f59-a787-18f0dc902e50 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:42.379422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138307e8-b15a-4f59-a787-18f0dc902e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:42.403411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-138307e8-b15a-4f59-a787-18f0dc902e50 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:42.407321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-138307e8-b15a-4f59-a787-18f0dc902e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:42.416508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1020/2013] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:42.420964 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ebb79-6ad4-4e7f-a393-3cbd2932f2cd None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:42.433510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ebb79-6ad4-4e7f-a393-3cbd2932f2cd None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:42.440335 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14c8e0-09ff-4847-8669-c91b4fb8b8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:42.440942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14c8e0-09ff-4847-8669-c91b4fb8b8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:42.441324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14c8e0-09ff-4847-8669-c91b4fb8b8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:42.441600 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14c8e0-09ff-4847-8669-c91b4fb8b8b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:42.442657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14c8e0-09ff-4847-8669-c91b4fb8b8b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:39:42.442922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14c8e0-09ff-4847-8669-c91b4fb8b8b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:42.444577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14c8e0-09ff-4847-8669-c91b4fb8b8b1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:42.460106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 994/2014] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:39:42.464370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138307e8-b15a-4f59-a787-18f0dc902e50 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:42.474755 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138307e8-b15a-4f59-a787-18f0dc902e50 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:42.480283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35903832-7983-4c7f-a0d6-c0645495ec7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:42.480906 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35903832-7983-4c7f-a0d6-c0645495ec7b admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:42.481220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35903832-7983-4c7f-a0d6-c0645495ec7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:42.481490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35903832-7983-4c7f-a0d6-c0645495ec7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:42.482132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35903832-7983-4c7f-a0d6-c0645495ec7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:42.485810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35903832-7983-4c7f-a0d6-c0645495ec7b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:42.539726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1021/2015] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:42 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:39:42.542014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14c8e0-09ff-4847-8669-c91b4fb8b8b1 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:42.552862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14c8e0-09ff-4847-8669-c91b4fb8b8b1 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:42.638240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35903832-7983-4c7f-a0d6-c0645495ec7b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:42.640037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-35903832-7983-4c7f-a0d6-c0645495ec7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:42.685612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288209d7-a36e-4df6-b131-099175e5da7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:42.686360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288209d7-a36e-4df6-b131-099175e5da7e admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:42.687206 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288209d7-a36e-4df6-b131-099175e5da7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:42.687595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288209d7-a36e-4df6-b131-099175e5da7e admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:42.689918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288209d7-a36e-4df6-b131-099175e5da7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:42.691955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288209d7-a36e-4df6-b131-099175e5da7e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:42.736880 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 995/2016] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:42 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:42.737837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288209d7-a36e-4df6-b131-099175e5da7e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:42.740342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-288209d7-a36e-4df6-b131-099175e5da7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:42.753500 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288209d7-a36e-4df6-b131-099175e5da7e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:42.758237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:42.758872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:42.759150 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:42.759418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] PATH_INFO: `/v3/projects/09c98239d7f3486ebc429aad02391120` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:42.760441 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35903832-7983-4c7f-a0d6-c0645495ec7b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:42.765347 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c98239d7f3486ebc429aad02391120)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:42.765496 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:42.766568 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:42.766846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:42.767112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] PATH_INFO: `/v3/users/e5b8a49a8fc44d59919e268088895bd0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:42.768355 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5b8a49a8fc44d59919e268088895bd0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:42.768787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:42.770724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:42.776604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:42.827788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5b8a49a8fc44d59919e268088895bd0'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.838568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c98239d7f3486ebc429aad02391120', 'request_id': 'req-f79d5337-946c-4956-baf8-8f872329c0e5'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.846818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c98239d7f3486ebc429aad02391120', 'request_id': 'req-f79d5337-946c-4956-baf8-8f872329c0e5'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.851169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5b8a49a8fc44d59919e268088895bd0'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.852518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5b8a49a8fc44d59919e268088895bd0'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.857890 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c98239d7f3486ebc429aad02391120', 'request_id': 'req-f79d5337-946c-4956-baf8-8f872329c0e5'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.862689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5b8a49a8fc44d59919e268088895bd0'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.863518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5b8a49a8fc44d59919e268088895bd0'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.876143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] The token cache is being invalidate because project 09c98239d7f3486ebc429aad02391120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:42.876969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.878544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:42.881318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 996/2017] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:42 2022] DELETE /identity/v3/projects/09c98239d7f3486ebc429aad02391120 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:42.882701 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1022/2018] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:42 2022] DELETE /identity/v3/users/e5b8a49a8fc44d59919e268088895bd0 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:42.885805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:42.887082 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:42.887678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:42.888595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:42.919149 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:42.970037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f96938-aafc-4f07-b14d-b76bb29034c1 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:42.970243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d5337-946c-4956-baf8-8f872329c0e5 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:42.975298 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe1dac-08e5-4cb5-8692-99e679d6c03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:42.975575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:42.976133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe1dac-08e5-4cb5-8692-99e679d6c03a admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:42.976241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:42.976402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe1dac-08e5-4cb5-8692-99e679d6c03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:42.976530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:42.976689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] PATH_INFO: `/v3/users/ee44fccf81784e78afcbccdb4d07223a` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:42.976825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe1dac-08e5-4cb5-8692-99e679d6c03a admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:42.977637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe1dac-08e5-4cb5-8692-99e679d6c03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:42.977938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee44fccf81784e78afcbccdb4d07223a)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:42.980293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe1dac-08e5-4cb5-8692-99e679d6c03a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:42.980456 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:42.994482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 997/2019] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:42.994482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b852fb9-c426-4126-9908-9983002bce38 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:42.994482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b852fb9-c426-4126-9908-9983002bce38 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:42.994482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b852fb9-c426-4126-9908-9983002bce38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:43.013916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-2b852fb9-c426-4126-9908-9983002bce38 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:43.015117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2b852fb9-c426-4126-9908-9983002bce38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:43.024393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 998/2020] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:43.026191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b852fb9-c426-4126-9908-9983002bce38 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:43.037849 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b852fb9-c426-4126-9908-9983002bce38 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:43.044120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab00dc0-c76a-4355-92e8-c804864fb3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:43.044812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab00dc0-c76a-4355-92e8-c804864fb3cc admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:43.045122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab00dc0-c76a-4355-92e8-c804864fb3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:43.045406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab00dc0-c76a-4355-92e8-c804864fb3cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:43.046053 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab00dc0-c76a-4355-92e8-c804864fb3cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:43.047940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee44fccf81784e78afcbccdb4d07223a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:43.048349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab00dc0-c76a-4355-92e8-c804864fb3cc admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:43.058942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 999/2021] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:43.061475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee44fccf81784e78afcbccdb4d07223a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:43.062370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab00dc0-c76a-4355-92e8-c804864fb3cc admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:43.062501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee44fccf81784e78afcbccdb4d07223a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:43.064983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3ab00dc0-c76a-4355-92e8-c804864fb3cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:43.069980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee44fccf81784e78afcbccdb4d07223a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:43.070650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee44fccf81784e78afcbccdb4d07223a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:43.093414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1023/2022] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:42 2022] DELETE /identity/v3/users/ee44fccf81784e78afcbccdb4d07223a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:43.093414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:43.106476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:43.137241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab00dc0-c76a-4355-92e8-c804864fb3cc admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:43.181654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:43.215759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bf76c-b42c-460f-a8d7-0be0eb2fabc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:43.216700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bf76c-b42c-460f-a8d7-0be0eb2fabc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:43.217076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bf76c-b42c-460f-a8d7-0be0eb2fabc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:43.217609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bf76c-b42c-460f-a8d7-0be0eb2fabc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:43.220700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d21bf76c-b42c-460f-a8d7-0be0eb2fabc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:43.224357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f90fa-6091-48a3-9b69-8e5594a483d7 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:43.231218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75124-5e64-4c8f-bece-0434c41f9cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:43.231868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75124-5e64-4c8f-bece-0434c41f9cd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:43.232253 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75124-5e64-4c8f-bece-0434c41f9cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:43.233449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75124-5e64-4c8f-bece-0434c41f9cd2 admin admin] PATH_INFO: `/v3/projects/38733a569d6d4bd885b814fb4d9ea6b4/users/ab83ace6325f49c29e0d1763997ab766/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:43.238143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bf76c-b42c-460f-a8d7-0be0eb2fabc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:43.239946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bf76c-b42c-460f-a8d7-0be0eb2fabc9 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:43.252473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d75124-5e64-4c8f-bece-0434c41f9cd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38733a569d6d4bd885b814fb4d9ea6b4, user_id=ab83ace6325f49c29e0d1763997ab766, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:43.255595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d75124-5e64-4c8f-bece-0434c41f9cd2 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:43.256628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1000/2023] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:43.276262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1024/2024] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:43 2022] PUT /identity/v3/projects/38733a569d6d4bd885b814fb4d9ea6b4/users/ab83ace6325f49c29e0d1763997ab766/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:43.289651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff15980-a3e0-4910-8d7c-79232dbdf28a None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:43.290276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff15980-a3e0-4910-8d7c-79232dbdf28a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:43.290680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff15980-a3e0-4910-8d7c-79232dbdf28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:43.327102 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-dff15980-a3e0-4910-8d7c-79232dbdf28a None None] MFA Rules not processed for user `ab83ace6325f49c29e0d1763997ab766`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:43.329315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-dff15980-a3e0-4910-8d7c-79232dbdf28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:43.420432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1001/2025] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:43.420756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75124-5e64-4c8f-bece-0434c41f9cd2 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:43.461677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:43.507927 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75124-5e64-4c8f-bece-0434c41f9cd2 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:43.513551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7844e-c81e-4391-8172-7047117f263a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:43.514308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7844e-c81e-4391-8172-7047117f263a admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:43.514861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7844e-c81e-4391-8172-7047117f263a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:43.515238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7844e-c81e-4391-8172-7047117f263a admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:43.516688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7844e-c81e-4391-8172-7047117f263a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:43.518787 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7844e-c81e-4391-8172-7047117f263a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:43.562564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1025/2026] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:43 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:43.564354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff15980-a3e0-4910-8d7c-79232dbdf28a None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:43.576455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff15980-a3e0-4910-8d7c-79232dbdf28a None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:43.681380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbe150-5ca6-44ee-969f-4845fa0d874d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:43.682154 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbe150-5ca6-44ee-969f-4845fa0d874d admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:43.682506 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbe150-5ca6-44ee-969f-4845fa0d874d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:43.682908 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbe150-5ca6-44ee-969f-4845fa0d874d admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:43.683636 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defbe150-5ca6-44ee-969f-4845fa0d874d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:43.686068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defbe150-5ca6-44ee-969f-4845fa0d874d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:43.711856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1002/2027] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:43 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:43.716749 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7844e-c81e-4391-8172-7047117f263a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:43.736638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7844e-c81e-4391-8172-7047117f263a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:43.745473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2a762-c4b5-437e-a5ee-d95797ec9e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:43.747100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2a762-c4b5-437e-a5ee-d95797ec9e87 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:43.748131 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2a762-c4b5-437e-a5ee-d95797ec9e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:43.748412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2a762-c4b5-437e-a5ee-d95797ec9e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:43.749237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2a762-c4b5-437e-a5ee-d95797ec9e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:43.752210 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2a762-c4b5-437e-a5ee-d95797ec9e87 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:43.774995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1026/2028] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:43.776749 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbe150-5ca6-44ee-969f-4845fa0d874d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:43.796483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbe150-5ca6-44ee-969f-4845fa0d874d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:43.804149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aef8ab-d731-4c34-8fe2-8d4914a40a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:43.806116 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aef8ab-d731-4c34-8fe2-8d4914a40a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:43.806604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aef8ab-d731-4c34-8fe2-8d4914a40a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:43.807061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aef8ab-d731-4c34-8fe2-8d4914a40a72 admin admin] PATH_INFO: `/v3/projects/0cd42fbe2d1240b79f306d1f85ad7c95/users/27d370beb0544e709b32ca7023556326/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:43.830241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aef8ab-d731-4c34-8fe2-8d4914a40a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd42fbe2d1240b79f306d1f85ad7c95, user_id=27d370beb0544e709b32ca7023556326, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:43.832845 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aef8ab-d731-4c34-8fe2-8d4914a40a72 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:43.835105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2a762-c4b5-437e-a5ee-d95797ec9e87 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:43.837015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-fda2a762-c4b5-437e-a5ee-d95797ec9e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:43.864582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1003/2029] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:43 2022] PUT /identity/v3/projects/0cd42fbe2d1240b79f306d1f85ad7c95/users/27d370beb0544e709b32ca7023556326/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:43.865372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2a762-c4b5-437e-a5ee-d95797ec9e87 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:43.866417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aef8ab-d731-4c34-8fe2-8d4914a40a72 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:43.909468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:43.938952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:43.939952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:43.940417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:43.941014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] PATH_INFO: `/v3/projects/34b00239804a45488ff8981c0e108f29` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:43.943004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34b00239804a45488ff8981c0e108f29)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:43.946234 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:43.948746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aef8ab-d731-4c34-8fe2-8d4914a40a72 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:43.956074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9742f44-4a06-47b7-bb3b-e6c33f41bf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:43.956451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9742f44-4a06-47b7-bb3b-e6c33f41bf38 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:43.957340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9742f44-4a06-47b7-bb3b-e6c33f41bf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:43.958455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9742f44-4a06-47b7-bb3b-e6c33f41bf38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:43.959221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9742f44-4a06-47b7-bb3b-e6c33f41bf38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:43.961070 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9742f44-4a06-47b7-bb3b-e6c33f41bf38 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:43.979695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1004/2030] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:43.984366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9742f44-4a06-47b7-bb3b-e6c33f41bf38 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:43.996343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9742f44-4a06-47b7-bb3b-e6c33f41bf38 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:44.006135 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34b00239804a45488ff8981c0e108f29', 'request_id': 'req-9005e17f-72e4-4ee6-9acc-cb648aa02011'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.015655 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34b00239804a45488ff8981c0e108f29', 'request_id': 'req-9005e17f-72e4-4ee6-9acc-cb648aa02011'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.023175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34b00239804a45488ff8981c0e108f29', 'request_id': 'req-9005e17f-72e4-4ee6-9acc-cb648aa02011'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.039471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] The token cache is being invalidate because project 34b00239804a45488ff8981c0e108f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:44.040190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.040789 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.042572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1027/2031] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:43 2022] DELETE /identity/v3/projects/34b00239804a45488ff8981c0e108f29 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:44.043495 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:44.044801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:44.083986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:44.092789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fcd084-cf43-40b6-af3d-b22c69ef016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:44.093503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcd084-cf43-40b6-af3d-b22c69ef016a admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:44.093797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcd084-cf43-40b6-af3d-b22c69ef016a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:44.094206 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcd084-cf43-40b6-af3d-b22c69ef016a admin admin] PATH_INFO: `/v3/projects/0cd42fbe2d1240b79f306d1f85ad7c95/users/27d370beb0544e709b32ca7023556326/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:44.096682 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fcd084-cf43-40b6-af3d-b22c69ef016a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd42fbe2d1240b79f306d1f85ad7c95, user_id=27d370beb0544e709b32ca7023556326, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:44.099597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fcd084-cf43-40b6-af3d-b22c69ef016a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:44.111601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1005/2032] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:43 2022] PUT /identity/v3/projects/0cd42fbe2d1240b79f306d1f85ad7c95/users/27d370beb0544e709b32ca7023556326/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:44.117999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fcd084-cf43-40b6-af3d-b22c69ef016a admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:44.122058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-86fcd084-cf43-40b6-af3d-b22c69ef016a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:44.129998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005e17f-72e4-4ee6-9acc-cb648aa02011 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:44.203281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:44.203955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:44.204233 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:44.204420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fcd084-cf43-40b6-af3d-b22c69ef016a admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:44.204525 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1 admin admin] PATH_INFO: `/v3/projects/aa45278fe2c4439c9b1015a34c41405e` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:44.205654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa45278fe2c4439c9b1015a34c41405e)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:44.207546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:44.209415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:44.210350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:44.211210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:44.211491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] PATH_INFO: `/v3/users/28132135d7244b5ba71b9c53fbcdadd6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:44.212695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28132135d7244b5ba71b9c53fbcdadd6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:44.214630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:44.262378 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa45278fe2c4439c9b1015a34c41405e', 'request_id': 'req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.270176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa45278fe2c4439c9b1015a34c41405e', 'request_id': 'req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.270527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28132135d7244b5ba71b9c53fbcdadd6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.277581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa45278fe2c4439c9b1015a34c41405e', 'request_id': 'req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.278650 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28132135d7244b5ba71b9c53fbcdadd6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.279393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28132135d7244b5ba71b9c53fbcdadd6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.280311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28132135d7244b5ba71b9c53fbcdadd6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.285908 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:44.292020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28132135d7244b5ba71b9c53fbcdadd6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.293566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1 admin admin] The token cache is being invalidate because project aa45278fe2c4439c9b1015a34c41405e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:44.293875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.294547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-59d1d9c9-a854-4807-a02e-563d3ad8c9b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.296317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1028/2033] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:44 2022] DELETE /identity/v3/projects/aa45278fe2c4439c9b1015a34c41405e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:44.297556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87a85b-f3f1-4a1d-92f7-34be8edbf8d3 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:44.297833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87a85b-f3f1-4a1d-92f7-34be8edbf8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:44.298151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87a85b-f3f1-4a1d-92f7-34be8edbf8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:44.315759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1006/2034] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:44 2022] DELETE /identity/v3/users/28132135d7244b5ba71b9c53fbcdadd6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:44.316763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:44.318348 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:44.334040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2a87a85b-f3f1-4a1d-92f7-34be8edbf8d3 None None] MFA Rules not processed for user `27d370beb0544e709b32ca7023556326`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:44.335604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2a87a85b-f3f1-4a1d-92f7-34be8edbf8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:44.393882 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:44.395325 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf45e8a-5965-46c1-bbac-d097565ddc20 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:44.401257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:44.402091 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:44.402389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:44.403489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] PATH_INFO: `/v3/users/a0ef95ca2e55488f83a13202382aa04c` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:44.403832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0ef95ca2e55488f83a13202382aa04c)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:44.405940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:44.440889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1029/2035] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:44.455880 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87a85b-f3f1-4a1d-92f7-34be8edbf8d3 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:44.458731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2a87a85b-f3f1-4a1d-92f7-34be8edbf8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:44.488556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0ef95ca2e55488f83a13202382aa04c'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.498780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0ef95ca2e55488f83a13202382aa04c'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.499851 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0ef95ca2e55488f83a13202382aa04c'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.501566 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0ef95ca2e55488f83a13202382aa04c'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.510992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0ef95ca2e55488f83a13202382aa04c'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:44.534823 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1007/2036] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:44 2022] DELETE /identity/v3/users/a0ef95ca2e55488f83a13202382aa04c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:44.578202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87a85b-f3f1-4a1d-92f7-34be8edbf8d3 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:44.669429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f8f1a-285d-4516-a906-b86891304784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:44.670059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f8f1a-285d-4516-a906-b86891304784 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:44.670359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f8f1a-285d-4516-a906-b86891304784 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:44.670637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f8f1a-285d-4516-a906-b86891304784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:44.672813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e24f8f1a-285d-4516-a906-b86891304784 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:44.739864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f8f1a-285d-4516-a906-b86891304784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:44.741448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f8f1a-285d-4516-a906-b86891304784 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:44.753150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1030/2037] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:39:44.912876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:44.915362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:45.066215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:45.072185 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a56a5-de8e-48df-8fbd-8e233081da59 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.079752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea10fcc-c487-4473-a207-94ae07703124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.081020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea10fcc-c487-4473-a207-94ae07703124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.081641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea10fcc-c487-4473-a207-94ae07703124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.082164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea10fcc-c487-4473-a207-94ae07703124 admin admin] PATH_INFO: `/v3/projects/c719c0c913704c76a8f8a7ff1f2dae62` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.082844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f8f1a-285d-4516-a906-b86891304784 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.084366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e24f8f1a-285d-4516-a906-b86891304784 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:45.084907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea10fcc-c487-4473-a207-94ae07703124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c719c0c913704c76a8f8a7ff1f2dae62)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.088037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea10fcc-c487-4473-a207-94ae07703124 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.113229 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f8f1a-285d-4516-a906-b86891304784 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.131854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b302c14-61ec-4f09-97a1-aa890857c20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.132953 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b302c14-61ec-4f09-97a1-aa890857c20e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.133285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b302c14-61ec-4f09-97a1-aa890857c20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.133543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b302c14-61ec-4f09-97a1-aa890857c20e admin admin] PATH_INFO: `/v3/projects/0b565c2cc25b4faf898dc29de7c5d908` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.135368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b302c14-61ec-4f09-97a1-aa890857c20e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b565c2cc25b4faf898dc29de7c5d908)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.137574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b302c14-61ec-4f09-97a1-aa890857c20e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.166649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bea10fcc-c487-4473-a207-94ae07703124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c719c0c913704c76a8f8a7ff1f2dae62', 'request_id': 'req-bea10fcc-c487-4473-a207-94ae07703124'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:45.177818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bea10fcc-c487-4473-a207-94ae07703124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c719c0c913704c76a8f8a7ff1f2dae62', 'request_id': 'req-bea10fcc-c487-4473-a207-94ae07703124'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:45.193045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bea10fcc-c487-4473-a207-94ae07703124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c719c0c913704c76a8f8a7ff1f2dae62', 'request_id': 'req-bea10fcc-c487-4473-a207-94ae07703124'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:45.212906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bea10fcc-c487-4473-a207-94ae07703124 admin admin] The token cache is being invalidate because project c719c0c913704c76a8f8a7ff1f2dae62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:45.213243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bea10fcc-c487-4473-a207-94ae07703124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:45.213952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bea10fcc-c487-4473-a207-94ae07703124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:45.216416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1008/2038] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:44 2022] DELETE /identity/v3/projects/c719c0c913704c76a8f8a7ff1f2dae62 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:45.224438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224792f5-68ab-46b3-971a-f606b6a9fbf7 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.224838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224792f5-68ab-46b3-971a-f606b6a9fbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.225259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224792f5-68ab-46b3-971a-f606b6a9fbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.226013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-5b302c14-61ec-4f09-97a1-aa890857c20e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b565c2cc25b4faf898dc29de7c5d908', 'request_id': 'req-5b302c14-61ec-4f09-97a1-aa890857c20e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:45.238096 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:45.239869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-5b302c14-61ec-4f09-97a1-aa890857c20e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b565c2cc25b4faf898dc29de7c5d908', 'request_id': 'req-5b302c14-61ec-4f09-97a1-aa890857c20e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:45.248939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-5b302c14-61ec-4f09-97a1-aa890857c20e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b565c2cc25b4faf898dc29de7c5d908', 'request_id': 'req-5b302c14-61ec-4f09-97a1-aa890857c20e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:45.249548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-224792f5-68ab-46b3-971a-f606b6a9fbf7 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:45.251427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-224792f5-68ab-46b3-971a-f606b6a9fbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:45.275276 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-5b302c14-61ec-4f09-97a1-aa890857c20e admin admin] The token cache is being invalidate because project 0b565c2cc25b4faf898dc29de7c5d908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:45.275580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-5b302c14-61ec-4f09-97a1-aa890857c20e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:45.276285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-5b302c14-61ec-4f09-97a1-aa890857c20e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:45.278066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1031/2039] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:45 2022] DELETE /identity/v3/projects/0b565c2cc25b4faf898dc29de7c5d908 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:45.285074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d313e70-99de-49d3-ad1a-1f89a6a7fe7f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.286002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d313e70-99de-49d3-ad1a-1f89a6a7fe7f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.286354 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d313e70-99de-49d3-ad1a-1f89a6a7fe7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.320097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-0d313e70-99de-49d3-ad1a-1f89a6a7fe7f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:45.320097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-0d313e70-99de-49d3-ad1a-1f89a6a7fe7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:45.383639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1009/2040] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:45.391827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0bfeb-5194-4220-b099-b2e8b80e7933 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.392111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0bfeb-5194-4220-b099-b2e8b80e7933 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.392378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0bfeb-5194-4220-b099-b2e8b80e7933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.411765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1032/2041] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:45.421788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3805a7-e680-47fe-892e-4e709aed9f14 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.422652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3805a7-e680-47fe-892e-4e709aed9f14 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.422937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3805a7-e680-47fe-892e-4e709aed9f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.425248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-78c0bfeb-5194-4220-b099-b2e8b80e7933 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:45.426869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-78c0bfeb-5194-4220-b099-b2e8b80e7933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:45.445917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1010/2042] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:45.446102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-1d3805a7-e680-47fe-892e-4e709aed9f14 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:45.446535 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1d3805a7-e680-47fe-892e-4e709aed9f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:45.447458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0bfeb-5194-4220-b099-b2e8b80e7933 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.457136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1033/2043] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:45.462991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3805a7-e680-47fe-892e-4e709aed9f14 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.466931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0bfeb-5194-4220-b099-b2e8b80e7933 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.473321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3805a7-e680-47fe-892e-4e709aed9f14 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.477325 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe309ec-5bdf-4dd6-ac0b-87e62a16dd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.478121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe309ec-5bdf-4dd6-ac0b-87e62a16dd3b admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.478586 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe309ec-5bdf-4dd6-ac0b-87e62a16dd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.478957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe309ec-5bdf-4dd6-ac0b-87e62a16dd3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.479346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0225f8a-9a4d-4ac8-bc9a-abd8379444db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.479814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe309ec-5bdf-4dd6-ac0b-87e62a16dd3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:39:45.480187 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe309ec-5bdf-4dd6-ac0b-87e62a16dd3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.480546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0225f8a-9a4d-4ac8-bc9a-abd8379444db admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.481003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0225f8a-9a4d-4ac8-bc9a-abd8379444db admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.481573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0225f8a-9a4d-4ac8-bc9a-abd8379444db admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.482409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0225f8a-9a4d-4ac8-bc9a-abd8379444db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:39:45.482914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0225f8a-9a4d-4ac8-bc9a-abd8379444db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.483051 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe309ec-5bdf-4dd6-ac0b-87e62a16dd3b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.484599 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0225f8a-9a4d-4ac8-bc9a-abd8379444db admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.496867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1011/2044] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:39:45.497216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1034/2045] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:39:45.501154 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe309ec-5bdf-4dd6-ac0b-87e62a16dd3b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.504248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0225f8a-9a4d-4ac8-bc9a-abd8379444db admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.512714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe309ec-5bdf-4dd6-ac0b-87e62a16dd3b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.515720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0225f8a-9a4d-4ac8-bc9a-abd8379444db admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.522952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbf4f0-0740-42bf-a852-c6b90ce56c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.523552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbf4f0-0740-42bf-a852-c6b90ce56c49 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.524230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbf4f0-0740-42bf-a852-c6b90ce56c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.524230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbf4f0-0740-42bf-a852-c6b90ce56c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.524712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbf4f0-0740-42bf-a852-c6b90ce56c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.527875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfd60a-3481-4171-936a-09c2d275de86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.528496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfd60a-3481-4171-936a-09c2d275de86 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.528768 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbf4f0-0740-42bf-a852-c6b90ce56c49 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.528876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfd60a-3481-4171-936a-09c2d275de86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.529033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfd60a-3481-4171-936a-09c2d275de86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.529713 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfd60a-3481-4171-936a-09c2d275de86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.532408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfd60a-3481-4171-936a-09c2d275de86 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.574199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1012/2046] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:45 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:39:45.580634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfd60a-3481-4171-936a-09c2d275de86 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.585670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1035/2047] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:45 2022] POST /identity/v3/projects => generated 371 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:39:45.588893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbf4f0-0740-42bf-a852-c6b90ce56c49 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.659614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:45.693384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfd60a-3481-4171-936a-09c2d275de86 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.702332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557d684-5725-4cc7-8aa7-3f67718b7488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.702882 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbf4f0-0740-42bf-a852-c6b90ce56c49 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.704063 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557d684-5725-4cc7-8aa7-3f67718b7488 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.704339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557d684-5725-4cc7-8aa7-3f67718b7488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.704614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557d684-5725-4cc7-8aa7-3f67718b7488 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.705442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557d684-5725-4cc7-8aa7-3f67718b7488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.707464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c557d684-5725-4cc7-8aa7-3f67718b7488 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.709083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee64f3-35e9-4f27-9099-631d315888b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.709811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee64f3-35e9-4f27-9099-631d315888b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.710170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee64f3-35e9-4f27-9099-631d315888b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.711127 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ee64f3-35e9-4f27-9099-631d315888b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.712009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee64f3-35e9-4f27-9099-631d315888b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.716091 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ee64f3-35e9-4f27-9099-631d315888b2 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.738282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1013/2048] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:45 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:45.742789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557d684-5725-4cc7-8aa7-3f67718b7488 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.750098 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1036/2049] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:45 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:45.754169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee64f3-35e9-4f27-9099-631d315888b2 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.757841 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557d684-5725-4cc7-8aa7-3f67718b7488 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.764014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8be71-8004-43a3-b167-df3c54ff565c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.764638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8be71-8004-43a3-b167-df3c54ff565c admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.764919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8be71-8004-43a3-b167-df3c54ff565c admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.765026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ee64f3-35e9-4f27-9099-631d315888b2 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.765233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8be71-8004-43a3-b167-df3c54ff565c admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.766661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8be71-8004-43a3-b167-df3c54ff565c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.768216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8be71-8004-43a3-b167-df3c54ff565c admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.770606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092ec4c-aecb-40d4-b65f-400a037d0752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.771417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092ec4c-aecb-40d4-b65f-400a037d0752 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.771771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092ec4c-aecb-40d4-b65f-400a037d0752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.772047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092ec4c-aecb-40d4-b65f-400a037d0752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.772700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092ec4c-aecb-40d4-b65f-400a037d0752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.774510 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092ec4c-aecb-40d4-b65f-400a037d0752 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.785184 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1014/2050] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:45.790376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8be71-8004-43a3-b167-df3c54ff565c admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.793316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1037/2051] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:39:45.797813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092ec4c-aecb-40d4-b65f-400a037d0752 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.800685 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8be71-8004-43a3-b167-df3c54ff565c admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.806833 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d9ef9-8513-4e31-8158-18eabd1aed70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.807460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d9ef9-8513-4e31-8158-18eabd1aed70 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.807807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d9ef9-8513-4e31-8158-18eabd1aed70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.808095 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d9ef9-8513-4e31-8158-18eabd1aed70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.808746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d9ef9-8513-4e31-8158-18eabd1aed70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.809301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092ec4c-aecb-40d4-b65f-400a037d0752 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.810422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d9ef9-8513-4e31-8158-18eabd1aed70 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.815278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fedb6-b4b0-430f-967e-7c4ce67ac49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.816040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fedb6-b4b0-430f-967e-7c4ce67ac49c admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.816393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fedb6-b4b0-430f-967e-7c4ce67ac49c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.816786 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fedb6-b4b0-430f-967e-7c4ce67ac49c admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.817493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fedb6-b4b0-430f-967e-7c4ce67ac49c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.819158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fedb6-b4b0-430f-967e-7c4ce67ac49c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.829445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1015/2052] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:45.830421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d9ef9-8513-4e31-8158-18eabd1aed70 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.832920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1038/2053] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:39:45.836660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fedb6-b4b0-430f-967e-7c4ce67ac49c admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:45.841304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d9ef9-8513-4e31-8158-18eabd1aed70 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.846834 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca305b-dd7b-4947-a9f4-bc3fa46845d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.847510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca305b-dd7b-4947-a9f4-bc3fa46845d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.847939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca305b-dd7b-4947-a9f4-bc3fa46845d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.848221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca305b-dd7b-4947-a9f4-bc3fa46845d7 admin admin] PATH_INFO: `/v3/projects/cc4cffd3a7134123a108619f69757edc/users/e32355437ce745a38d4756c73782c1d0/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.852917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fedb6-b4b0-430f-967e-7c4ce67ac49c admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:45.861502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a4bb0-73e1-4da9-a305-b44f9a5e7e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:45.862308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a4bb0-73e1-4da9-a305-b44f9a5e7e0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.862659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a4bb0-73e1-4da9-a305-b44f9a5e7e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.863010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a4bb0-73e1-4da9-a305-b44f9a5e7e0b admin admin] PATH_INFO: `/v3/projects/ecf3d02df950406f8530cb3ab89f2de9/users/58590e92355c4c2d9a6cbd0a7cf81338/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.872282 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:45.877620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca305b-dd7b-4947-a9f4-bc3fa46845d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc4cffd3a7134123a108619f69757edc, user_id=e32355437ce745a38d4756c73782c1d0, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.880991 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca305b-dd7b-4947-a9f4-bc3fa46845d7 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.881187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a4bb0-73e1-4da9-a305-b44f9a5e7e0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecf3d02df950406f8530cb3ab89f2de9, user_id=58590e92355c4c2d9a6cbd0a7cf81338, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:45.884350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a4bb0-73e1-4da9-a305-b44f9a5e7e0b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:45.898106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1016/2054] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:45 2022] PUT /identity/v3/projects/cc4cffd3a7134123a108619f69757edc/users/e32355437ce745a38d4756c73782c1d0/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:45.912410 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a217877-3d42-4649-9af1-08b1dc1a8db5 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.912608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1039/2055] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:45 2022] PUT /identity/v3/projects/ecf3d02df950406f8530cb3ab89f2de9/users/58590e92355c4c2d9a6cbd0a7cf81338/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:45.912703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a217877-3d42-4649-9af1-08b1dc1a8db5 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.912703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a217877-3d42-4649-9af1-08b1dc1a8db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.914832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6dcd3-0654-4c60-b0cd-8a7dbbb16fb5 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:45.915375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6dcd3-0654-4c60-b0cd-8a7dbbb16fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:45.915652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6dcd3-0654-4c60-b0cd-8a7dbbb16fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:45.943096 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-24c6dcd3-0654-4c60-b0cd-8a7dbbb16fb5 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:45.943362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-2a217877-3d42-4649-9af1-08b1dc1a8db5 None None] MFA Rules not processed for user `e32355437ce745a38d4756c73782c1d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:45.944467 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-24c6dcd3-0654-4c60-b0cd-8a7dbbb16fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:45.944674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2a217877-3d42-4649-9af1-08b1dc1a8db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:46.026466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1017/2056] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:46.038281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a217877-3d42-4649-9af1-08b1dc1a8db5 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:46.039509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1040/2057] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:39:46.040160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2a217877-3d42-4649-9af1-08b1dc1a8db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:46.050003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6dcd3-0654-4c60-b0cd-8a7dbbb16fb5 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:46.051440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-24c6dcd3-0654-4c60-b0cd-8a7dbbb16fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:46.107736 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a217877-3d42-4649-9af1-08b1dc1a8db5 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:46.112080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f317f-c1f6-4c0c-95f2-97b763924e43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:46.112697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f317f-c1f6-4c0c-95f2-97b763924e43 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:46.113032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f317f-c1f6-4c0c-95f2-97b763924e43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:46.113344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f317f-c1f6-4c0c-95f2-97b763924e43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:46.116202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6dcd3-0654-4c60-b0cd-8a7dbbb16fb5 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:46.119849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b9e79-5ceb-47a3-9995-40bb481f6333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:46.120470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b9e79-5ceb-47a3-9995-40bb481f6333 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:46.120749 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b9e79-5ceb-47a3-9995-40bb481f6333 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:46.121022 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b9e79-5ceb-47a3-9995-40bb481f6333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:46.123129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f317f-c1f6-4c0c-95f2-97b763924e43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:46.124613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f317f-c1f6-4c0c-95f2-97b763924e43 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:46.132933 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b9e79-5ceb-47a3-9995-40bb481f6333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:46.134729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b9e79-5ceb-47a3-9995-40bb481f6333 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:46.135743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1018/2058] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:39:46.146340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1041/2059] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:47.069019 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eefb0-91ca-4650-990d-77d461b16fd5 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:47.069577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eefb0-91ca-4650-990d-77d461b16fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:47.069577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eefb0-91ca-4650-990d-77d461b16fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:47.108694 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-650eefb0-91ca-4650-990d-77d461b16fd5 None None] MFA Rules not processed for user `58590e92355c4c2d9a6cbd0a7cf81338`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:47.110545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-650eefb0-91ca-4650-990d-77d461b16fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:47.121557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:47.215335 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1019/2060] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:47.219856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b9e79-5ceb-47a3-9995-40bb481f6333 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:47.232309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b9e79-5ceb-47a3-9995-40bb481f6333 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:47.238959 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463318e-d708-4ed8-b99f-cb64d3464f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:47.239588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463318e-d708-4ed8-b99f-cb64d3464f7c admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:47.239907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463318e-d708-4ed8-b99f-cb64d3464f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:47.240183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463318e-d708-4ed8-b99f-cb64d3464f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:47.240889 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463318e-d708-4ed8-b99f-cb64d3464f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:47.243508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463318e-d708-4ed8-b99f-cb64d3464f7c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:47.282262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1042/2061] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:47 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:39:47.285991 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eefb0-91ca-4650-990d-77d461b16fd5 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:47.299582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eefb0-91ca-4650-990d-77d461b16fd5 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:47.363881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cfe24-5738-4613-aa77-fcb3c8acc472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:47.364508 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cfe24-5738-4613-aa77-fcb3c8acc472 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:47.364816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cfe24-5738-4613-aa77-fcb3c8acc472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:47.365138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cfe24-5738-4613-aa77-fcb3c8acc472 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:47.365765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cfe24-5738-4613-aa77-fcb3c8acc472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:47.367873 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cfe24-5738-4613-aa77-fcb3c8acc472 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:47.395918 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1020/2062] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:47 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:39:47.396661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463318e-d708-4ed8-b99f-cb64d3464f7c admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:47.408166 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463318e-d708-4ed8-b99f-cb64d3464f7c admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:47.413477 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a16cc-a5c5-4744-a67f-6c993d5e3acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:47.414420 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a16cc-a5c5-4744-a67f-6c993d5e3acc admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:47.414770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a16cc-a5c5-4744-a67f-6c993d5e3acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:47.415362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a16cc-a5c5-4744-a67f-6c993d5e3acc admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:47.416091 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a16cc-a5c5-4744-a67f-6c993d5e3acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:47.418846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a16cc-a5c5-4744-a67f-6c993d5e3acc admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:47.431921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1043/2063] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:39:47.436277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cfe24-5738-4613-aa77-fcb3c8acc472 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:47.446910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cfe24-5738-4613-aa77-fcb3c8acc472 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:47.451678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a2e1f-a972-4b2e-9088-a8122c6910e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:47.452339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a2e1f-a972-4b2e-9088-a8122c6910e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:47.452620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a2e1f-a972-4b2e-9088-a8122c6910e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:47.452894 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a2e1f-a972-4b2e-9088-a8122c6910e6 admin admin] PATH_INFO: `/v3/projects/28f9c06434b644519ae34a991cb581e2/users/d2e6ca5993ce499397eff5b7fffd8687/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:47.468786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a2e1f-a972-4b2e-9088-a8122c6910e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f9c06434b644519ae34a991cb581e2, user_id=d2e6ca5993ce499397eff5b7fffd8687, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:47.471994 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a2e1f-a972-4b2e-9088-a8122c6910e6 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:47.485580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1021/2064] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:47 2022] PUT /identity/v3/projects/28f9c06434b644519ae34a991cb581e2/users/d2e6ca5993ce499397eff5b7fffd8687/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:39:47.489715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a16cc-a5c5-4744-a67f-6c993d5e3acc admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:47.499804 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a16cc-a5c5-4744-a67f-6c993d5e3acc admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:47.554387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:47.561401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0482c839-72d2-49b1-87b6-ef37b5210fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:47.562018 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482c839-72d2-49b1-87b6-ef37b5210fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:47.562527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482c839-72d2-49b1-87b6-ef37b5210fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:47.563057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0482c839-72d2-49b1-87b6-ef37b5210fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:47.563695 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482c839-72d2-49b1-87b6-ef37b5210fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:47.565299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0482c839-72d2-49b1-87b6-ef37b5210fd5 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:47.575911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1044/2065] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:47.580279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a2e1f-a972-4b2e-9088-a8122c6910e6 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:47.590951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a2e1f-a972-4b2e-9088-a8122c6910e6 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:47.597591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c71d2-4502-496e-bbaf-2a29be36d764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:47.598262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c71d2-4502-496e-bbaf-2a29be36d764 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:47.598536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c71d2-4502-496e-bbaf-2a29be36d764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:47.598806 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c71d2-4502-496e-bbaf-2a29be36d764 admin admin] PATH_INFO: `/v3/projects/28f9c06434b644519ae34a991cb581e2/users/d2e6ca5993ce499397eff5b7fffd8687/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:47.600711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c71d2-4502-496e-bbaf-2a29be36d764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f9c06434b644519ae34a991cb581e2, user_id=d2e6ca5993ce499397eff5b7fffd8687, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:47.603569 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c71d2-4502-496e-bbaf-2a29be36d764 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:47.615648 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1022/2066] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:47 2022] PUT /identity/v3/projects/28f9c06434b644519ae34a991cb581e2/users/d2e6ca5993ce499397eff5b7fffd8687/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:48.338937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b038a30-8240-4292-906b-1dc82070bc87 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:48.339478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b038a30-8240-4292-906b-1dc82070bc87 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:48.339478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b038a30-8240-4292-906b-1dc82070bc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:48.377776 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2b038a30-8240-4292-906b-1dc82070bc87 None None] MFA Rules not processed for user `d2e6ca5993ce499397eff5b7fffd8687`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:48.379148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2b038a30-8240-4292-906b-1dc82070bc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:48.504235 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1045/2067] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:48.518123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c71d2-4502-496e-bbaf-2a29be36d764 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:48.519983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1f3c71d2-4502-496e-bbaf-2a29be36d764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:48.598543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c71d2-4502-496e-bbaf-2a29be36d764 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:48.604012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64079816-f279-4b06-8299-1453c7132185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:48.605166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64079816-f279-4b06-8299-1453c7132185 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:48.605166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64079816-f279-4b06-8299-1453c7132185 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:48.605939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64079816-f279-4b06-8299-1453c7132185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:48.622431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:48.691108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64079816-f279-4b06-8299-1453c7132185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:48.692943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64079816-f279-4b06-8299-1453c7132185 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:48.703212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1023/2068] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:49.256011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b038a30-8240-4292-906b-1dc82070bc87 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:49.258720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2b038a30-8240-4292-906b-1dc82070bc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:49.386981 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b038a30-8240-4292-906b-1dc82070bc87 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:49.396866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:49.399583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:49.399817 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:49.400089 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] PATH_INFO: `/v3/users/ab83ace6325f49c29e0d1763997ab766` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:49.401479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab83ace6325f49c29e0d1763997ab766)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:49.403805 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:49.472486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab83ace6325f49c29e0d1763997ab766'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:49.487643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab83ace6325f49c29e0d1763997ab766'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:49.488735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab83ace6325f49c29e0d1763997ab766'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:49.497865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab83ace6325f49c29e0d1763997ab766'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:49.498619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab83ace6325f49c29e0d1763997ab766'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:49.513361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:49.528994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1046/2069] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:49 2022] DELETE /identity/v3/users/ab83ace6325f49c29e0d1763997ab766 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:49.534725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afccb1-a21f-49e0-b424-b37590ffe891 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:49.535504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afccb1-a21f-49e0-b424-b37590ffe891 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:49.535674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afccb1-a21f-49e0-b424-b37590ffe891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:49.557058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-e1afccb1-a21f-49e0-b424-b37590ffe891 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:49.558392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e1afccb1-a21f-49e0-b424-b37590ffe891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:49.624861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1024/2070] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:49.639018 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:49.640626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:49.700864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22311757-1023-4dec-8d03-aaa671e106fd admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:49.705306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad533d-d5d1-4be4-b606-75523bb6b4e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:49.705918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad533d-d5d1-4be4-b606-75523bb6b4e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:49.706198 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad533d-d5d1-4be4-b606-75523bb6b4e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:49.706484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad533d-d5d1-4be4-b606-75523bb6b4e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:49.716812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad533d-d5d1-4be4-b606-75523bb6b4e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:49.718272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad533d-d5d1-4be4-b606-75523bb6b4e8 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:49.727941 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1047/2071] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:50.017604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afccb1-a21f-49e0-b424-b37590ffe891 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:50.019061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e1afccb1-a21f-49e0-b424-b37590ffe891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:50.031764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afccb1-a21f-49e0-b424-b37590ffe891 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:50.037393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:50.038056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:50.038336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:50.038617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] PATH_INFO: `/v3/projects/38733a569d6d4bd885b814fb4d9ea6b4` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:50.039797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38733a569d6d4bd885b814fb4d9ea6b4)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:50.042033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:50.097251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38733a569d6d4bd885b814fb4d9ea6b4', 'request_id': 'req-d788ad42-24e1-4134-840c-181acfe5f216'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:50.104571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38733a569d6d4bd885b814fb4d9ea6b4', 'request_id': 'req-d788ad42-24e1-4134-840c-181acfe5f216'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:50.113985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38733a569d6d4bd885b814fb4d9ea6b4', 'request_id': 'req-d788ad42-24e1-4134-840c-181acfe5f216'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:50.128446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] The token cache is being invalidate because project 38733a569d6d4bd885b814fb4d9ea6b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:50.128906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:50.130008 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:50.131718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1025/2072] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:50 2022] DELETE /identity/v3/projects/38733a569d6d4bd885b814fb4d9ea6b4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:50.141188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad533d-d5d1-4be4-b606-75523bb6b4e8 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:50.162995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:50.164535 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:50.173292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad533d-d5d1-4be4-b606-75523bb6b4e8 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:50.222642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:50.241766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788ad42-24e1-4134-840c-181acfe5f216 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:50.252561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095415b-499a-49ca-8e94-2b23b72fbb51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:50.253339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095415b-499a-49ca-8e94-2b23b72fbb51 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:50.253694 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095415b-499a-49ca-8e94-2b23b72fbb51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:50.254042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095415b-499a-49ca-8e94-2b23b72fbb51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:50.256599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1095415b-499a-49ca-8e94-2b23b72fbb51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:50.309549 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75edb6-ffec-4d36-8272-833d67d6d7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:50.310567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75edb6-ffec-4d36-8272-833d67d6d7ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:50.311160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75edb6-ffec-4d36-8272-833d67d6d7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:50.311501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75edb6-ffec-4d36-8272-833d67d6d7ac admin admin] PATH_INFO: `/v3/users/27d370beb0544e709b32ca7023556326` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:50.313332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75edb6-ffec-4d36-8272-833d67d6d7ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d370beb0544e709b32ca7023556326)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:50.315422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75edb6-ffec-4d36-8272-833d67d6d7ac admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:50.355268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095415b-499a-49ca-8e94-2b23b72fbb51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:50.357204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095415b-499a-49ca-8e94-2b23b72fbb51 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:50.371499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1026/2073] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:39:50.409909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bd75edb6-ffec-4d36-8272-833d67d6d7ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d370beb0544e709b32ca7023556326'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:50.430767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bd75edb6-ffec-4d36-8272-833d67d6d7ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d370beb0544e709b32ca7023556326'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:50.432400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bd75edb6-ffec-4d36-8272-833d67d6d7ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d370beb0544e709b32ca7023556326'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:50.447382 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bd75edb6-ffec-4d36-8272-833d67d6d7ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d370beb0544e709b32ca7023556326'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:50.448205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bd75edb6-ffec-4d36-8272-833d67d6d7ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d370beb0544e709b32ca7023556326'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:50.471170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1048/2074] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:50 2022] DELETE /identity/v3/users/27d370beb0544e709b32ca7023556326 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:50.851823 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095415b-499a-49ca-8e94-2b23b72fbb51 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:50.854121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1095415b-499a-49ca-8e94-2b23b72fbb51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:50.884083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095415b-499a-49ca-8e94-2b23b72fbb51 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:50.982619 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:50.983745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:50.984542 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:50.985297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3 admin admin] PATH_INFO: `/v3/projects/0cd42fbe2d1240b79f306d1f85ad7c95` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:50.987730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd42fbe2d1240b79f306d1f85ad7c95)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:50.989984 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:51.055354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd42fbe2d1240b79f306d1f85ad7c95', 'request_id': 'req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:51.061201 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd42fbe2d1240b79f306d1f85ad7c95', 'request_id': 'req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:51.071032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd42fbe2d1240b79f306d1f85ad7c95', 'request_id': 'req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:51.085548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3 admin admin] The token cache is being invalidate because project 0cd42fbe2d1240b79f306d1f85ad7c95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:51.086082 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:51.087628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-97f79f80-95b6-4255-b30e-4a2eaf8c3bc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:51.090776 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1027/2075] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:50 2022] DELETE /identity/v3/projects/0cd42fbe2d1240b79f306d1f85ad7c95 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:51.095997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbe218-d29e-4868-b970-0dfbb386e09b None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:51.096369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbe218-d29e-4868-b970-0dfbb386e09b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:51.096716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbe218-d29e-4868-b970-0dfbb386e09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:51.117311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-21dbe218-d29e-4868-b970-0dfbb386e09b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:51.118643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-21dbe218-d29e-4868-b970-0dfbb386e09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:51.133581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:51.190150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1049/2076] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:51.198883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad6ea9-ecab-4b38-9318-5074ad54e973 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:51.199293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad6ea9-ecab-4b38-9318-5074ad54e973 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:51.199712 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad6ea9-ecab-4b38-9318-5074ad54e973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:51.221873 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-69ad6ea9-ecab-4b38-9318-5074ad54e973 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:51.223112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-69ad6ea9-ecab-4b38-9318-5074ad54e973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:51.231899 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1028/2077] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:51.236462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbe218-d29e-4868-b970-0dfbb386e09b None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:51.246297 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbe218-d29e-4868-b970-0dfbb386e09b None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:51.251774 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035e221-a46b-4c54-911c-daaac6f48853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:51.252388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e221-a46b-4c54-911c-daaac6f48853 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:51.252671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e221-a46b-4c54-911c-daaac6f48853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:51.252945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e221-a46b-4c54-911c-daaac6f48853 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:51.253661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035e221-a46b-4c54-911c-daaac6f48853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:39:51.254010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035e221-a46b-4c54-911c-daaac6f48853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:51.255555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035e221-a46b-4c54-911c-daaac6f48853 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:51.269345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1050/2078] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:39:51.274028 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad6ea9-ecab-4b38-9318-5074ad54e973 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:51.283747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad6ea9-ecab-4b38-9318-5074ad54e973 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:51.289396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c2819-a085-4cb9-aca4-2a0fb23b9f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:51.290324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c2819-a085-4cb9-aca4-2a0fb23b9f8f admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:51.290603 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c2819-a085-4cb9-aca4-2a0fb23b9f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:51.290875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c2819-a085-4cb9-aca4-2a0fb23b9f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:51.291536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c2819-a085-4cb9-aca4-2a0fb23b9f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:51.293539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c2819-a085-4cb9-aca4-2a0fb23b9f8f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:51.331337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:51.363466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1029/2079] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:51 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:51.366385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035e221-a46b-4c54-911c-daaac6f48853 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:51.377819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035e221-a46b-4c54-911c-daaac6f48853 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:51.457414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecd5a3-7bcc-453a-bd15-51a01ca95e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:51.458127 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecd5a3-7bcc-453a-bd15-51a01ca95e6b admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:51.458368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecd5a3-7bcc-453a-bd15-51a01ca95e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:51.458648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecd5a3-7bcc-453a-bd15-51a01ca95e6b admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:51.459371 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecd5a3-7bcc-453a-bd15-51a01ca95e6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:51.461988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecd5a3-7bcc-453a-bd15-51a01ca95e6b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:51.499940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1051/2080] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:51 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:51.501639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c2819-a085-4cb9-aca4-2a0fb23b9f8f admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:51.514690 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c2819-a085-4cb9-aca4-2a0fb23b9f8f admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:51.522913 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcaba33-5003-4d7f-b168-a87f28a60e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:51.523608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcaba33-5003-4d7f-b168-a87f28a60e5b admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:51.523954 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcaba33-5003-4d7f-b168-a87f28a60e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:51.524293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcaba33-5003-4d7f-b168-a87f28a60e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:51.525038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcaba33-5003-4d7f-b168-a87f28a60e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:51.527567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcaba33-5003-4d7f-b168-a87f28a60e5b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:51.553332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1030/2081] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:51.553458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecd5a3-7bcc-453a-bd15-51a01ca95e6b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:51.565995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecd5a3-7bcc-453a-bd15-51a01ca95e6b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:51.575815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d23839-90ba-4644-aaf8-90774a2b7d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:51.576438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d23839-90ba-4644-aaf8-90774a2b7d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:51.576718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d23839-90ba-4644-aaf8-90774a2b7d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:51.577142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d23839-90ba-4644-aaf8-90774a2b7d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:51.577867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d23839-90ba-4644-aaf8-90774a2b7d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:51.579485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d23839-90ba-4644-aaf8-90774a2b7d2c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:51.594184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1052/2082] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:51.598394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcaba33-5003-4d7f-b168-a87f28a60e5b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:51.614947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcaba33-5003-4d7f-b168-a87f28a60e5b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:51.620306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bcb7b-0424-401b-a35c-c45d03bba83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:51.621411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bcb7b-0424-401b-a35c-c45d03bba83d admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:51.621692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bcb7b-0424-401b-a35c-c45d03bba83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:51.622237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bcb7b-0424-401b-a35c-c45d03bba83d admin admin] PATH_INFO: `/v3/projects/edf617f972c94683aa992967533699eb/users/6c8f89e7420c4cb3b26d42285d65855e/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:51.649650 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bcb7b-0424-401b-a35c-c45d03bba83d admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf617f972c94683aa992967533699eb, user_id=6c8f89e7420c4cb3b26d42285d65855e, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:51.652445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bcb7b-0424-401b-a35c-c45d03bba83d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:51.684581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1031/2083] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:51 2022] PUT /identity/v3/projects/edf617f972c94683aa992967533699eb/users/6c8f89e7420c4cb3b26d42285d65855e/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:51.689913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea92ad-6b3c-4d13-b0f8-96c54faf7970 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:51.690218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea92ad-6b3c-4d13-b0f8-96c54faf7970 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:51.690487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea92ad-6b3c-4d13-b0f8-96c54faf7970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:51.723579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-65ea92ad-6b3c-4d13-b0f8-96c54faf7970 None None] MFA Rules not processed for user `6c8f89e7420c4cb3b26d42285d65855e`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:51.724814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-65ea92ad-6b3c-4d13-b0f8-96c54faf7970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:51.839129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1053/2084] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:51.907455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bcb7b-0424-401b-a35c-c45d03bba83d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:51.909437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-0a8bcb7b-0424-401b-a35c-c45d03bba83d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:51.982493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bcb7b-0424-401b-a35c-c45d03bba83d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:51.991923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79073b-d383-4dd0-9feb-acee91fff360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:51.992598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79073b-d383-4dd0-9feb-acee91fff360 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:51.992927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79073b-d383-4dd0-9feb-acee91fff360 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:51.993247 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79073b-d383-4dd0-9feb-acee91fff360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:52.003023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79073b-d383-4dd0-9feb-acee91fff360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:52.004463 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79073b-d383-4dd0-9feb-acee91fff360 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:52.019775 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1032/2085] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:53.821032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea92ad-6b3c-4d13-b0f8-96c54faf7970 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:53.822473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-65ea92ad-6b3c-4d13-b0f8-96c54faf7970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:53.841115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:53.891755 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea92ad-6b3c-4d13-b0f8-96c54faf7970 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:53.896567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:53.897231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:53.897509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:53.900938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] PATH_INFO: `/v3/users/ec01a5f046e24da5b917fbecdc05a2f8` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:53.902616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec01a5f046e24da5b917fbecdc05a2f8)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:53.904673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:53.952524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec01a5f046e24da5b917fbecdc05a2f8'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:53.965631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec01a5f046e24da5b917fbecdc05a2f8'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:53.966312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec01a5f046e24da5b917fbecdc05a2f8'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:53.974325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec01a5f046e24da5b917fbecdc05a2f8'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:53.974979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec01a5f046e24da5b917fbecdc05a2f8'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:53.989407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1054/2086] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:53 2022] DELETE /identity/v3/users/ec01a5f046e24da5b917fbecdc05a2f8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:53.993813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27b7ca-46bc-4416-8410-9309fe63706b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:53.994362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27b7ca-46bc-4416-8410-9309fe63706b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:53.994702 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27b7ca-46bc-4416-8410-9309fe63706b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:54.014767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-cb27b7ca-46bc-4416-8410-9309fe63706b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:54.015982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-cb27b7ca-46bc-4416-8410-9309fe63706b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:54.076881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1033/2087] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:54.086623 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:54.088081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:54.149679 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64e998-4f11-4e93-8f5a-0cdea07b5eb1 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:54.153643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01bd54-295f-41ac-97fe-116c464c350e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:54.154264 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01bd54-295f-41ac-97fe-116c464c350e service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:54.154584 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01bd54-295f-41ac-97fe-116c464c350e service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:54.154860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01bd54-295f-41ac-97fe-116c464c350e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:54.166871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01bd54-295f-41ac-97fe-116c464c350e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:54.168292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01bd54-295f-41ac-97fe-116c464c350e service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:54.180131 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1055/2088] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:54.537752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27b7ca-46bc-4416-8410-9309fe63706b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:54.539980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-cb27b7ca-46bc-4416-8410-9309fe63706b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:54.558761 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27b7ca-46bc-4416-8410-9309fe63706b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:54.567438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:54.568182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:54.568503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:54.568875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28 admin admin] PATH_INFO: `/v3/projects/cd3e119c3abc473786626a27c865300e` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:54.570446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd3e119c3abc473786626a27c865300e)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:54.574052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:54.631692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd3e119c3abc473786626a27c865300e', 'request_id': 'req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:54.636068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:54.639061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd3e119c3abc473786626a27c865300e', 'request_id': 'req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:54.648584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd3e119c3abc473786626a27c865300e', 'request_id': 'req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:54.661598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28 admin admin] The token cache is being invalidate because project cd3e119c3abc473786626a27c865300e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:54.661922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:54.662930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c0093f93-ba8d-4dcc-be4f-6d6fae84dc28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:54.664820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1034/2089] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:54 2022] DELETE /identity/v3/projects/cd3e119c3abc473786626a27c865300e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:54.670049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b764e-259c-449c-a4fc-dc2be62bd770 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:54.670336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b764e-259c-449c-a4fc-dc2be62bd770 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:54.670613 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b764e-259c-449c-a4fc-dc2be62bd770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:54.690628 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-920b764e-259c-449c-a4fc-dc2be62bd770 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:54.691947 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-920b764e-259c-449c-a4fc-dc2be62bd770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:54.726263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:54.759388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1056/2090] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:54.767644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee84a89-02c0-4c25-9455-bbb46ab14028 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:54.767927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee84a89-02c0-4c25-9455-bbb46ab14028 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:54.768200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee84a89-02c0-4c25-9455-bbb46ab14028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:54.793489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-fee84a89-02c0-4c25-9455-bbb46ab14028 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:54.794950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-fee84a89-02c0-4c25-9455-bbb46ab14028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:54.803361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1035/2091] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:54.807679 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b764e-259c-449c-a4fc-dc2be62bd770 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:54.818256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b764e-259c-449c-a4fc-dc2be62bd770 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:54.823041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587bf59-5692-4676-82b0-5b3ab6250e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:54.823648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587bf59-5692-4676-82b0-5b3ab6250e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:54.824108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587bf59-5692-4676-82b0-5b3ab6250e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:54.824385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587bf59-5692-4676-82b0-5b3ab6250e80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:54.825064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587bf59-5692-4676-82b0-5b3ab6250e80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:39:54.825476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587bf59-5692-4676-82b0-5b3ab6250e80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:54.827094 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587bf59-5692-4676-82b0-5b3ab6250e80 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:54.840539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1057/2092] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:39:54.846572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee84a89-02c0-4c25-9455-bbb46ab14028 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:54.856000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee84a89-02c0-4c25-9455-bbb46ab14028 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:54.860862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62cea6-e809-4ad9-8828-7fee6c6959c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:54.861523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62cea6-e809-4ad9-8828-7fee6c6959c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:54.861806 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62cea6-e809-4ad9-8828-7fee6c6959c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:54.862079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62cea6-e809-4ad9-8828-7fee6c6959c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:54.862727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62cea6-e809-4ad9-8828-7fee6c6959c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:54.864634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62cea6-e809-4ad9-8828-7fee6c6959c2 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:54.898193 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1036/2093] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:39:54 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:39:54.899342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587bf59-5692-4676-82b0-5b3ab6250e80 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:54.910842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587bf59-5692-4676-82b0-5b3ab6250e80 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:54.982595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba020c-7df4-437f-b7dd-d4582efe76e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:54.983294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba020c-7df4-437f-b7dd-d4582efe76e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:54.983576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba020c-7df4-437f-b7dd-d4582efe76e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:54.983904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba020c-7df4-437f-b7dd-d4582efe76e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:54.984549 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba020c-7df4-437f-b7dd-d4582efe76e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:54.989963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba020c-7df4-437f-b7dd-d4582efe76e7 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:55.015918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1058/2094] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:39:54 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:39:55.019929 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62cea6-e809-4ad9-8828-7fee6c6959c2 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:55.032554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62cea6-e809-4ad9-8828-7fee6c6959c2 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:55.042368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edd1ce-6fe0-4369-8e1e-528fcf62fe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:55.042985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edd1ce-6fe0-4369-8e1e-528fcf62fe51 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:55.043267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edd1ce-6fe0-4369-8e1e-528fcf62fe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:55.043599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edd1ce-6fe0-4369-8e1e-528fcf62fe51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:55.044189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edd1ce-6fe0-4369-8e1e-528fcf62fe51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:55.045865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edd1ce-6fe0-4369-8e1e-528fcf62fe51 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:55.058501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1037/2095] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:55.063339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba020c-7df4-437f-b7dd-d4582efe76e7 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:55.075782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba020c-7df4-437f-b7dd-d4582efe76e7 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:55.082516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e4777-9c79-47fd-961a-024412abfacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:55.083142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e4777-9c79-47fd-961a-024412abfacb admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:55.083424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e4777-9c79-47fd-961a-024412abfacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:55.084226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e4777-9c79-47fd-961a-024412abfacb admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:55.084353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e4777-9c79-47fd-961a-024412abfacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:55.085967 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e4777-9c79-47fd-961a-024412abfacb admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:55.101591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1059/2096] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:39:55.102977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edd1ce-6fe0-4369-8e1e-528fcf62fe51 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:55.114678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edd1ce-6fe0-4369-8e1e-528fcf62fe51 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:55.119614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1b6a3-ba66-4be7-a87a-3a6845ea370d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:55.120490 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1b6a3-ba66-4be7-a87a-3a6845ea370d admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:55.120776 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1b6a3-ba66-4be7-a87a-3a6845ea370d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:55.121055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1b6a3-ba66-4be7-a87a-3a6845ea370d admin admin] PATH_INFO: `/v3/projects/3777d8561b4e4c9385e87007b37fa57a/users/5c45623c9421460abfa049a8040c7c12/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:55.137182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1b6a3-ba66-4be7-a87a-3a6845ea370d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3777d8561b4e4c9385e87007b37fa57a, user_id=5c45623c9421460abfa049a8040c7c12, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:55.139951 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1b6a3-ba66-4be7-a87a-3a6845ea370d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:55.154887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1038/2097] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:39:55 2022] PUT /identity/v3/projects/3777d8561b4e4c9385e87007b37fa57a/users/5c45623c9421460abfa049a8040c7c12/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:55.165761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c40d81-771c-4a64-a17f-4a59f3d9557b None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:55.166047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c40d81-771c-4a64-a17f-4a59f3d9557b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:55.166317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c40d81-771c-4a64-a17f-4a59f3d9557b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:55.196145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-08c40d81-771c-4a64-a17f-4a59f3d9557b None None] MFA Rules not processed for user `5c45623c9421460abfa049a8040c7c12`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:55.197397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-08c40d81-771c-4a64-a17f-4a59f3d9557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:55.269063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1060/2098] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:55.286290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1b6a3-ba66-4be7-a87a-3a6845ea370d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:55.287979 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-eae1b6a3-ba66-4be7-a87a-3a6845ea370d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:55.369215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1b6a3-ba66-4be7-a87a-3a6845ea370d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:55.373081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408aef0d-05dd-4815-9841-1deaef2c454a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:55.373727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408aef0d-05dd-4815-9841-1deaef2c454a service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:55.374010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408aef0d-05dd-4815-9841-1deaef2c454a service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:55.374283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408aef0d-05dd-4815-9841-1deaef2c454a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:55.383501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408aef0d-05dd-4815-9841-1deaef2c454a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:55.384942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408aef0d-05dd-4815-9841-1deaef2c454a service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:55.394199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1039/2099] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:57.905109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3905ea-5565-42b3-899b-bcee09b9ed69 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:57.905691 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3905ea-5565-42b3-899b-bcee09b9ed69 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:57.905691 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3905ea-5565-42b3-899b-bcee09b9ed69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:57.933004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-0d3905ea-5565-42b3-899b-bcee09b9ed69 None None] MFA Rules not processed for user `5c45623c9421460abfa049a8040c7c12`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:57.934556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-0d3905ea-5565-42b3-899b-bcee09b9ed69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:57.943728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1061/2100] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:58.842654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408aef0d-05dd-4815-9841-1deaef2c454a service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:58.844752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-408aef0d-05dd-4815-9841-1deaef2c454a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:58.917281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408aef0d-05dd-4815-9841-1deaef2c454a service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:58.921859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:58.922668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:58.922948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:58.923249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] PATH_INFO: `/v3/users/6c8f89e7420c4cb3b26d42285d65855e` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:58.924375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c8f89e7420c4cb3b26d42285d65855e)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:58.926356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:58.948251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:58.975057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c8f89e7420c4cb3b26d42285d65855e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:58.981841 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c8f89e7420c4cb3b26d42285d65855e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:58.982538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c8f89e7420c4cb3b26d42285d65855e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:58.983144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c8f89e7420c4cb3b26d42285d65855e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:58.990706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c8f89e7420c4cb3b26d42285d65855e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:59.011103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1040/2101] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:39:58 2022] DELETE /identity/v3/users/6c8f89e7420c4cb3b26d42285d65855e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:39:59.015539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a883e9-613f-4977-a0e8-581a80eb022a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:59.016511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a883e9-613f-4977-a0e8-581a80eb022a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:59.016616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a883e9-613f-4977-a0e8-581a80eb022a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:59.037022 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b0a883e9-613f-4977-a0e8-581a80eb022a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:39:59.038307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b0a883e9-613f-4977-a0e8-581a80eb022a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:59.060254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:39:59.116271 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1062/2102] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:39:59.128126 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:59.129999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:59.223390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81c701-ba31-4f23-a6c6-ecfb99368ced admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:59.230995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6c78d-7628-4483-9d25-ce05b316758c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:59.232914 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6c78d-7628-4483-9d25-ce05b316758c service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:59.233292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6c78d-7628-4483-9d25-ce05b316758c service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:59.233589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6c78d-7628-4483-9d25-ce05b316758c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:59.248678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6c78d-7628-4483-9d25-ce05b316758c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:59.257267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6c78d-7628-4483-9d25-ce05b316758c service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:59.265038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1041/2103] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:39:59.772257 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a883e9-613f-4977-a0e8-581a80eb022a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:39:59.773914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b0a883e9-613f-4977-a0e8-581a80eb022a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:39:59.797301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a883e9-613f-4977-a0e8-581a80eb022a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:39:59.803590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035ffe6-8827-4ce7-a012-bca605b38ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:39:59.805242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035ffe6-8827-4ce7-a012-bca605b38ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:39:59.805909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035ffe6-8827-4ce7-a012-bca605b38ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:39:59.817170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035ffe6-8827-4ce7-a012-bca605b38ba1 admin admin] PATH_INFO: `/v3/projects/edf617f972c94683aa992967533699eb` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:39:59.819218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035ffe6-8827-4ce7-a012-bca605b38ba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf617f972c94683aa992967533699eb)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:39:59.821523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035ffe6-8827-4ce7-a012-bca605b38ba1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:39:59.913958 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-5035ffe6-8827-4ce7-a012-bca605b38ba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf617f972c94683aa992967533699eb', 'request_id': 'req-5035ffe6-8827-4ce7-a012-bca605b38ba1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:59.928613 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-5035ffe6-8827-4ce7-a012-bca605b38ba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf617f972c94683aa992967533699eb', 'request_id': 'req-5035ffe6-8827-4ce7-a012-bca605b38ba1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:59.937959 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-5035ffe6-8827-4ce7-a012-bca605b38ba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf617f972c94683aa992967533699eb', 'request_id': 'req-5035ffe6-8827-4ce7-a012-bca605b38ba1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:59.985872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-5035ffe6-8827-4ce7-a012-bca605b38ba1 admin admin] The token cache is being invalidate because project edf617f972c94683aa992967533699eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:39:59.986332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-5035ffe6-8827-4ce7-a012-bca605b38ba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:59.989637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-5035ffe6-8827-4ce7-a012-bca605b38ba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:39:59.995923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1063/2104] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:39:59 2022] DELETE /identity/v3/projects/edf617f972c94683aa992967533699eb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:00.011476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92222cb-1e72-4b1d-9041-0a7af97088bd None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:00.012147 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92222cb-1e72-4b1d-9041-0a7af97088bd None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:00.012147 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92222cb-1e72-4b1d-9041-0a7af97088bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:00.049658 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-d92222cb-1e72-4b1d-9041-0a7af97088bd None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:00.051012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d92222cb-1e72-4b1d-9041-0a7af97088bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:00.136663 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1042/2105] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:00.146008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94930b9-d1fd-4aaa-a03f-ea6174d7a756 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:00.146008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94930b9-d1fd-4aaa-a03f-ea6174d7a756 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:00.146608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94930b9-d1fd-4aaa-a03f-ea6174d7a756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:00.173809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-a94930b9-d1fd-4aaa-a03f-ea6174d7a756 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:00.175473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a94930b9-d1fd-4aaa-a03f-ea6174d7a756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:00.188513 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1064/2106] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:00.192433 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92222cb-1e72-4b1d-9041-0a7af97088bd None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:00.204612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92222cb-1e72-4b1d-9041-0a7af97088bd None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:00.214323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2a21e-9766-4d83-afea-c617d4332df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:00.215045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2a21e-9766-4d83-afea-c617d4332df6 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:00.215399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2a21e-9766-4d83-afea-c617d4332df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:00.215747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2a21e-9766-4d83-afea-c617d4332df6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:00.216486 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2a21e-9766-4d83-afea-c617d4332df6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:40:00.216867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2a21e-9766-4d83-afea-c617d4332df6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:00.218623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2a21e-9766-4d83-afea-c617d4332df6 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:00.235290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1043/2107] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:40:00.239905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94930b9-d1fd-4aaa-a03f-ea6174d7a756 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:00.251607 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94930b9-d1fd-4aaa-a03f-ea6174d7a756 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:00.257954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c903b4-ad05-47da-8632-a148960e4bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:00.262023 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c903b4-ad05-47da-8632-a148960e4bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:00.262556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c903b4-ad05-47da-8632-a148960e4bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:00.263731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c903b4-ad05-47da-8632-a148960e4bf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:00.264565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c903b4-ad05-47da-8632-a148960e4bf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:00.268646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c903b4-ad05-47da-8632-a148960e4bf0 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:00.335383 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1065/2108] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:40:00 2022] POST /identity/v3/projects => generated 363 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:40:00.337278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2a21e-9766-4d83-afea-c617d4332df6 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:00.350527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c903b4-ad05-47da-8632-a148960e4bf0 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:00.354849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-67c903b4-ad05-47da-8632-a148960e4bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:00.360570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2a21e-9766-4d83-afea-c617d4332df6 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:00.415054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:00.469405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940a05f-47a9-43b6-af74-aa7b6c64f3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:00.470039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940a05f-47a9-43b6-af74-aa7b6c64f3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:00.470323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940a05f-47a9-43b6-af74-aa7b6c64f3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:00.471734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940a05f-47a9-43b6-af74-aa7b6c64f3e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:00.472392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940a05f-47a9-43b6-af74-aa7b6c64f3e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:00.474568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940a05f-47a9-43b6-af74-aa7b6c64f3e8 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:00.486863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c903b4-ad05-47da-8632-a148960e4bf0 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:00.511486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51663721-2dee-4a8c-8203-f6c25f817302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:00.512406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51663721-2dee-4a8c-8203-f6c25f817302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:00.512688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51663721-2dee-4a8c-8203-f6c25f817302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:00.513010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51663721-2dee-4a8c-8203-f6c25f817302 admin admin] PATH_INFO: `/v3/users/e32355437ce745a38d4756c73782c1d0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:00.527077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1044/2109] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:40:00 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:40:00.528892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940a05f-47a9-43b6-af74-aa7b6c64f3e8 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:00.530573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51663721-2dee-4a8c-8203-f6c25f817302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e32355437ce745a38d4756c73782c1d0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:00.532753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51663721-2dee-4a8c-8203-f6c25f817302 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:00.555901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940a05f-47a9-43b6-af74-aa7b6c64f3e8 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:00.568059 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e9d2e-ef21-41a3-96c1-604009706b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:00.568672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e9d2e-ef21-41a3-96c1-604009706b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:00.569178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e9d2e-ef21-41a3-96c1-604009706b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:00.569470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e9d2e-ef21-41a3-96c1-604009706b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:00.571434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396e9d2e-ef21-41a3-96c1-604009706b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:00.577853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396e9d2e-ef21-41a3-96c1-604009706b7d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:00.586943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:00.609999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1045/2110] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:00.613182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e9d2e-ef21-41a3-96c1-604009706b7d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:00.647734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e9d2e-ef21-41a3-96c1-604009706b7d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:00.651946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-51663721-2dee-4a8c-8203-f6c25f817302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e32355437ce745a38d4756c73782c1d0'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:00.652950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3484ec1-6a0e-494d-90bc-0c6e0145abc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:00.653610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3484ec1-6a0e-494d-90bc-0c6e0145abc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:00.653889 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3484ec1-6a0e-494d-90bc-0c6e0145abc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:00.654638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3484ec1-6a0e-494d-90bc-0c6e0145abc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:00.655131 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3484ec1-6a0e-494d-90bc-0c6e0145abc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:00.657388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3484ec1-6a0e-494d-90bc-0c6e0145abc8 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:00.669257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1046/2111] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:00.672858 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-51663721-2dee-4a8c-8203-f6c25f817302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e32355437ce745a38d4756c73782c1d0'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:00.673167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3484ec1-6a0e-494d-90bc-0c6e0145abc8 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:00.673664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-51663721-2dee-4a8c-8203-f6c25f817302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e32355437ce745a38d4756c73782c1d0'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:00.675373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b3484ec1-6a0e-494d-90bc-0c6e0145abc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:00.689875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-51663721-2dee-4a8c-8203-f6c25f817302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e32355437ce745a38d4756c73782c1d0'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:00.690086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-51663721-2dee-4a8c-8203-f6c25f817302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e32355437ce745a38d4756c73782c1d0'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:00.693787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3484ec1-6a0e-494d-90bc-0c6e0145abc8 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:00.699476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1a014-42d7-4580-829f-49b2cafdfd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:00.700280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1a014-42d7-4580-829f-49b2cafdfd9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:00.700540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1a014-42d7-4580-829f-49b2cafdfd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:00.701001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1a014-42d7-4580-829f-49b2cafdfd9f admin admin] PATH_INFO: `/v3/projects/f07839378483494990a49ffdea7dd95e/users/2c43b1f9964f476bb63c51568779f08e/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:00.708505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1066/2112] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:40:00 2022] DELETE /identity/v3/users/e32355437ce745a38d4756c73782c1d0 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:00.723060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1a014-42d7-4580-829f-49b2cafdfd9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f07839378483494990a49ffdea7dd95e, user_id=2c43b1f9964f476bb63c51568779f08e, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:00.725474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2662a2-9324-41a1-b8b2-045593071b99 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:00.725857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2662a2-9324-41a1-b8b2-045593071b99 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:00.726142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2662a2-9324-41a1-b8b2-045593071b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:00.726602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1a014-42d7-4580-829f-49b2cafdfd9f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:00.758800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-1b2662a2-9324-41a1-b8b2-045593071b99 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:00.760838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1b2662a2-9324-41a1-b8b2-045593071b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:00.762957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1047/2113] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:40:00 2022] PUT /identity/v3/projects/f07839378483494990a49ffdea7dd95e/users/2c43b1f9964f476bb63c51568779f08e/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:40:00.777210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30148793-9a3a-4a9d-a4f4-c8ecb2ea2307 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:00.777398 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30148793-9a3a-4a9d-a4f4-c8ecb2ea2307 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:00.777716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30148793-9a3a-4a9d-a4f4-c8ecb2ea2307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:00.861431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-30148793-9a3a-4a9d-a4f4-c8ecb2ea2307 None None] MFA Rules not processed for user `2c43b1f9964f476bb63c51568779f08e`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:00.864188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-30148793-9a3a-4a9d-a4f4-c8ecb2ea2307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:00.867394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1067/2114] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:00.879419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2662a2-9324-41a1-b8b2-045593071b99 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:00.881988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1b2662a2-9324-41a1-b8b2-045593071b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:00.982079 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2662a2-9324-41a1-b8b2-045593071b99 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:00.989193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43dcff-d2bf-4ab8-863d-715281e2f68d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:00.989825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1048/2115] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:40:00.990205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43dcff-d2bf-4ab8-863d-715281e2f68d service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:00.990715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43dcff-d2bf-4ab8-863d-715281e2f68d service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:00.991162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43dcff-d2bf-4ab8-863d-715281e2f68d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:01.003172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30148793-9a3a-4a9d-a4f4-c8ecb2ea2307 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:01.005575 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-30148793-9a3a-4a9d-a4f4-c8ecb2ea2307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:01.006944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43dcff-d2bf-4ab8-863d-715281e2f68d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:01.008669 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43dcff-d2bf-4ab8-863d-715281e2f68d service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:01.025568 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1068/2116] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:40:01.084181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30148793-9a3a-4a9d-a4f4-c8ecb2ea2307 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:01.089296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc64742-d3d1-43ef-8793-4a75cfffc7b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:01.089921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc64742-d3d1-43ef-8793-4a75cfffc7b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:01.090946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc64742-d3d1-43ef-8793-4a75cfffc7b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:01.091402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc64742-d3d1-43ef-8793-4a75cfffc7b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:01.107768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc64742-d3d1-43ef-8793-4a75cfffc7b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:01.109602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc64742-d3d1-43ef-8793-4a75cfffc7b2 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:01.122354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1049/2117] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:40:01.720745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43dcff-d2bf-4ab8-863d-715281e2f68d service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:01.722345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-be43dcff-d2bf-4ab8-863d-715281e2f68d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:01.737712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43dcff-d2bf-4ab8-863d-715281e2f68d service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:01.742883 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:01.743523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:01.743806 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:01.744080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] PATH_INFO: `/v3/projects/cc4cffd3a7134123a108619f69757edc` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:01.745245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc4cffd3a7134123a108619f69757edc)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:01.747176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:01.801529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:01.836877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc4cffd3a7134123a108619f69757edc', 'request_id': 'req-dac1ca55-5ac4-4460-b13c-0f01cad88f00'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:01.849600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc4cffd3a7134123a108619f69757edc', 'request_id': 'req-dac1ca55-5ac4-4460-b13c-0f01cad88f00'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:01.860914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc4cffd3a7134123a108619f69757edc', 'request_id': 'req-dac1ca55-5ac4-4460-b13c-0f01cad88f00'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:01.876129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] The token cache is being invalidate because project cc4cffd3a7134123a108619f69757edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:40:01.876432 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:01.877129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:01.878628 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1069/2118] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:40:01 2022] DELETE /identity/v3/projects/cc4cffd3a7134123a108619f69757edc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:07.231050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc64742-d3d1-43ef-8793-4a75cfffc7b2 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:07.233015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1cc64742-d3d1-43ef-8793-4a75cfffc7b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:07.257905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc64742-d3d1-43ef-8793-4a75cfffc7b2 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:07.261813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:07.329947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:07.330759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:07.331310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:07.331771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] PATH_INFO: `/v3/users/58590e92355c4c2d9a6cbd0a7cf81338` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:07.342193 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58590e92355c4c2d9a6cbd0a7cf81338)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:07.345216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:07.403309 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58590e92355c4c2d9a6cbd0a7cf81338'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:07.410968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58590e92355c4c2d9a6cbd0a7cf81338'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:07.411653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58590e92355c4c2d9a6cbd0a7cf81338'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:07.412287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58590e92355c4c2d9a6cbd0a7cf81338'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:07.421504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58590e92355c4c2d9a6cbd0a7cf81338'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:07.442655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1050/2119] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:40:07 2022] DELETE /identity/v3/users/58590e92355c4c2d9a6cbd0a7cf81338 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:07.722421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:07.724046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:07.746326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1ca55-5ac4-4460-b13c-0f01cad88f00 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:07.846639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:07.847925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:07.848483 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:07.848910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] PATH_INFO: `/v3/projects/ecf3d02df950406f8530cb3ab89f2de9` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:07.850444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecf3d02df950406f8530cb3ab89f2de9)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:07.852591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:07.971130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecf3d02df950406f8530cb3ab89f2de9', 'request_id': 'req-06232500-0018-42f2-aec4-c50afa8aa692'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:07.979032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecf3d02df950406f8530cb3ab89f2de9', 'request_id': 'req-06232500-0018-42f2-aec4-c50afa8aa692'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:07.986607 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecf3d02df950406f8530cb3ab89f2de9', 'request_id': 'req-06232500-0018-42f2-aec4-c50afa8aa692'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:08.006106 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] The token cache is being invalidate because project ecf3d02df950406f8530cb3ab89f2de9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:40:08.006408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:08.007077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:08.010146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1070/2120] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:40:07 2022] DELETE /identity/v3/projects/ecf3d02df950406f8530cb3ab89f2de9 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:08.012683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:08.014307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:08.037931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3a1a2-e656-48d3-87dd-a7e1749ea073 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:08.183325 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c44064-384f-4ec2-9c20-fa2f9e251c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:08.184053 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c44064-384f-4ec2-9c20-fa2f9e251c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:08.184406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c44064-384f-4ec2-9c20-fa2f9e251c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:08.184748 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c44064-384f-4ec2-9c20-fa2f9e251c7e admin admin] PATH_INFO: `/v3/users/d2e6ca5993ce499397eff5b7fffd8687` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:08.186222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c44064-384f-4ec2-9c20-fa2f9e251c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e6ca5993ce499397eff5b7fffd8687)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:08.188295 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c44064-384f-4ec2-9c20-fa2f9e251c7e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:08.263537 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:08.295674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c7c44064-384f-4ec2-9c20-fa2f9e251c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e6ca5993ce499397eff5b7fffd8687'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:08.313641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c7c44064-384f-4ec2-9c20-fa2f9e251c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e6ca5993ce499397eff5b7fffd8687'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:08.315878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c7c44064-384f-4ec2-9c20-fa2f9e251c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e6ca5993ce499397eff5b7fffd8687'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:08.316920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c7c44064-384f-4ec2-9c20-fa2f9e251c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e6ca5993ce499397eff5b7fffd8687'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:08.334492 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c7c44064-384f-4ec2-9c20-fa2f9e251c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e6ca5993ce499397eff5b7fffd8687'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:08.374511 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1051/2121] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:40:08 2022] DELETE /identity/v3/users/d2e6ca5993ce499397eff5b7fffd8687 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:08.858331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:08.861421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:08.878871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06232500-0018-42f2-aec4-c50afa8aa692 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:08.942711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:08.953564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3bc52-d56f-4361-9e17-9e2645b82dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:08.955122 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3bc52-d56f-4361-9e17-9e2645b82dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:08.955848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3bc52-d56f-4361-9e17-9e2645b82dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:08.956172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3bc52-d56f-4361-9e17-9e2645b82dc1 admin admin] PATH_INFO: `/v3/projects/28f9c06434b644519ae34a991cb581e2` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:08.957674 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3bc52-d56f-4361-9e17-9e2645b82dc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28f9c06434b644519ae34a991cb581e2)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:08.960507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3bc52-d56f-4361-9e17-9e2645b82dc1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:09.015473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-1de3bc52-d56f-4361-9e17-9e2645b82dc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28f9c06434b644519ae34a991cb581e2', 'request_id': 'req-1de3bc52-d56f-4361-9e17-9e2645b82dc1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:09.022407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-1de3bc52-d56f-4361-9e17-9e2645b82dc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28f9c06434b644519ae34a991cb581e2', 'request_id': 'req-1de3bc52-d56f-4361-9e17-9e2645b82dc1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:09.029302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-1de3bc52-d56f-4361-9e17-9e2645b82dc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28f9c06434b644519ae34a991cb581e2', 'request_id': 'req-1de3bc52-d56f-4361-9e17-9e2645b82dc1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:09.044629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-1de3bc52-d56f-4361-9e17-9e2645b82dc1 admin admin] The token cache is being invalidate because project 28f9c06434b644519ae34a991cb581e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:40:09.045570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-1de3bc52-d56f-4361-9e17-9e2645b82dc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:09.046214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-1de3bc52-d56f-4361-9e17-9e2645b82dc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:09.048357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1071/2122] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:40:08 2022] DELETE /identity/v3/projects/28f9c06434b644519ae34a991cb581e2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:40:09.053368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22897529-3123-4cfa-b0b0-524f8e9d6b4a None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:09.053610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22897529-3123-4cfa-b0b0-524f8e9d6b4a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:09.053883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22897529-3123-4cfa-b0b0-524f8e9d6b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:09.079418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-22897529-3123-4cfa-b0b0-524f8e9d6b4a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:09.080632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-22897529-3123-4cfa-b0b0-524f8e9d6b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:09.160439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1052/2123] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:09.168232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5a0bf-a650-42d5-983f-bad8428d8677 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:09.168520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5a0bf-a650-42d5-983f-bad8428d8677 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:09.168828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5a0bf-a650-42d5-983f-bad8428d8677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:09.201841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-88b5a0bf-a650-42d5-983f-bad8428d8677 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:09.206731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-88b5a0bf-a650-42d5-983f-bad8428d8677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:09.217186 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1072/2124] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:09.221374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22897529-3123-4cfa-b0b0-524f8e9d6b4a None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:09.233141 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22897529-3123-4cfa-b0b0-524f8e9d6b4a None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:09.239149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64442e66-17db-461f-9c65-afa7cd91dde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:09.239773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64442e66-17db-461f-9c65-afa7cd91dde2 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:09.240054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64442e66-17db-461f-9c65-afa7cd91dde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:09.240328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64442e66-17db-461f-9c65-afa7cd91dde2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:09.240983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64442e66-17db-461f-9c65-afa7cd91dde2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:40:09.241331 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64442e66-17db-461f-9c65-afa7cd91dde2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:09.242931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64442e66-17db-461f-9c65-afa7cd91dde2 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:09.262474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5a0bf-a650-42d5-983f-bad8428d8677 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:09.266266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1053/2125] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:40:09.274447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5a0bf-a650-42d5-983f-bad8428d8677 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:09.280432 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add72c85-d64a-4b54-ada1-7be119614c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:09.280764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add72c85-d64a-4b54-ada1-7be119614c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:09.281002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add72c85-d64a-4b54-ada1-7be119614c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:09.281871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add72c85-d64a-4b54-ada1-7be119614c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:09.282693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add72c85-d64a-4b54-ada1-7be119614c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:09.284629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add72c85-d64a-4b54-ada1-7be119614c3f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:09.363193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1073/2126] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:40:09 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:40:09.363880 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64442e66-17db-461f-9c65-afa7cd91dde2 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:09.376780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64442e66-17db-461f-9c65-afa7cd91dde2 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:09.466216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb60471-cd09-40b5-aa9f-20a9e60106ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:09.467203 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb60471-cd09-40b5-aa9f-20a9e60106ff admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:09.467573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb60471-cd09-40b5-aa9f-20a9e60106ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:09.467947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb60471-cd09-40b5-aa9f-20a9e60106ff admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:09.468729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb60471-cd09-40b5-aa9f-20a9e60106ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:09.471272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb60471-cd09-40b5-aa9f-20a9e60106ff admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:09.496999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1054/2127] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:40:09 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:40:09.498270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add72c85-d64a-4b54-ada1-7be119614c3f admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:09.508754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add72c85-d64a-4b54-ada1-7be119614c3f admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:09.513429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6effd-c512-44a7-84ea-b5d4b6b18f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:09.514049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6effd-c512-44a7-84ea-b5d4b6b18f2f admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:09.514331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6effd-c512-44a7-84ea-b5d4b6b18f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:09.514605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6effd-c512-44a7-84ea-b5d4b6b18f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:09.515505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6effd-c512-44a7-84ea-b5d4b6b18f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:09.517689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6effd-c512-44a7-84ea-b5d4b6b18f2f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:09.528687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1074/2128] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:09.533372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb60471-cd09-40b5-aa9f-20a9e60106ff admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:09.543652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb60471-cd09-40b5-aa9f-20a9e60106ff admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:09.548251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c8a57-5bb0-41bb-b107-c196e103ff96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:09.548879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c8a57-5bb0-41bb-b107-c196e103ff96 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:09.549196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c8a57-5bb0-41bb-b107-c196e103ff96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:09.549475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c8a57-5bb0-41bb-b107-c196e103ff96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:09.550148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c8a57-5bb0-41bb-b107-c196e103ff96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:09.551740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c8a57-5bb0-41bb-b107-c196e103ff96 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:09.563122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1055/2129] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:09.567414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6effd-c512-44a7-84ea-b5d4b6b18f2f admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:09.577883 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6effd-c512-44a7-84ea-b5d4b6b18f2f admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:09.583549 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a94294-0e39-428f-aa36-acc98d68c3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:09.584311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a94294-0e39-428f-aa36-acc98d68c3f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:09.584592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a94294-0e39-428f-aa36-acc98d68c3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:09.584911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a94294-0e39-428f-aa36-acc98d68c3f0 admin admin] PATH_INFO: `/v3/projects/cd5d9c471dfd4410925e45a1b2585e0b/users/f29f67533d864a57ae8d7b4f564b514d/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:09.606995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a94294-0e39-428f-aa36-acc98d68c3f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd5d9c471dfd4410925e45a1b2585e0b, user_id=f29f67533d864a57ae8d7b4f564b514d, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:09.607944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a94294-0e39-428f-aa36-acc98d68c3f0 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:09.623095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1075/2130] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:40:09 2022] PUT /identity/v3/projects/cd5d9c471dfd4410925e45a1b2585e0b/users/f29f67533d864a57ae8d7b4f564b514d/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:40:09.634730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11e4a0-2f2c-4038-8b53-163d11541f3e None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:09.634982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11e4a0-2f2c-4038-8b53-163d11541f3e None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:09.635204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11e4a0-2f2c-4038-8b53-163d11541f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:09.662293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:09.698204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ab11e4a0-2f2c-4038-8b53-163d11541f3e None None] MFA Rules not processed for user `f29f67533d864a57ae8d7b4f564b514d`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:09.700640 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ab11e4a0-2f2c-4038-8b53-163d11541f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:09.838143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1056/2131] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:09.850932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a94294-0e39-428f-aa36-acc98d68c3f0 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:09.852880 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-16a94294-0e39-428f-aa36-acc98d68c3f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:09.928072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a94294-0e39-428f-aa36-acc98d68c3f0 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:09.931917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4db7d-f1e5-466f-a504-e4c16291e9f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:09.932655 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4db7d-f1e5-466f-a504-e4c16291e9f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:09.933036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4db7d-f1e5-466f-a504-e4c16291e9f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:09.933627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4db7d-f1e5-466f-a504-e4c16291e9f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:09.944609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4db7d-f1e5-466f-a504-e4c16291e9f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:09.946292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4db7d-f1e5-466f-a504-e4c16291e9f7 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:09.957136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1076/2132] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:40:09.989285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11e4a0-2f2c-4038-8b53-163d11541f3e None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:10.112878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11e4a0-2f2c-4038-8b53-163d11541f3e None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:10.126873 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:10.127886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:10.128278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:10.129049 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] PATH_INFO: `/v3/users/f29f67533d864a57ae8d7b4f564b514d` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:10.130847 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f29f67533d864a57ae8d7b4f564b514d)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:10.134854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:10.215599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f29f67533d864a57ae8d7b4f564b514d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:10.227300 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f29f67533d864a57ae8d7b4f564b514d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:10.228197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f29f67533d864a57ae8d7b4f564b514d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:10.229211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f29f67533d864a57ae8d7b4f564b514d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:10.243968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f29f67533d864a57ae8d7b4f564b514d'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:10.271958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1057/2133] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:40:09 2022] DELETE /identity/v3/users/f29f67533d864a57ae8d7b4f564b514d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:10.277082 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bff2f-973a-43b1-8494-4f822a50ddc6 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:10.277482 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bff2f-973a-43b1-8494-4f822a50ddc6 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:10.277821 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bff2f-973a-43b1-8494-4f822a50ddc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:10.307408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-7e4bff2f-973a-43b1-8494-4f822a50ddc6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:10.308808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7e4bff2f-973a-43b1-8494-4f822a50ddc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:10.342376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:10.390461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1077/2134] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:10.402733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:10.404255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:10.443161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:10.498857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f803d27-513f-4db6-8ceb-4cff56fca81f admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:10.502577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4054b3-537e-4da6-a002-e0c588b68e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:10.503199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4054b3-537e-4da6-a002-e0c588b68e5c service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:10.503478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4054b3-537e-4da6-a002-e0c588b68e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:10.503753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4054b3-537e-4da6-a002-e0c588b68e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:10.518799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4054b3-537e-4da6-a002-e0c588b68e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:10.520424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4054b3-537e-4da6-a002-e0c588b68e5c service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:10.534412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1058/2135] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:40:10.896965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041529dc-e5a9-4450-ad51-923bff26722b None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:10.897382 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041529dc-e5a9-4450-ad51-923bff26722b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:10.897746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041529dc-e5a9-4450-ad51-923bff26722b None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:10.899114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1078/2136] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:40:10.929906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2fc9a-6f9c-46ef-b4d6-823fecf9e198 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:10.930290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2fc9a-6f9c-46ef-b4d6-823fecf9e198 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:10.930550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2fc9a-6f9c-46ef-b4d6-823fecf9e198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:10.950704 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-cbe2fc9a-6f9c-46ef-b4d6-823fecf9e198 None None] MFA Rules not processed for user `6b40ee263df74e58bf3f958c01eb742f`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:10.951928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-cbe2fc9a-6f9c-46ef-b4d6-823fecf9e198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:11.009000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041529dc-e5a9-4450-ad51-923bff26722b None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:11.011177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-041529dc-e5a9-4450-ad51-923bff26722b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:11.036516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041529dc-e5a9-4450-ad51-923bff26722b None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:11.044970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29f3ce-d806-4856-a5a3-12f6d069c821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:11.045676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29f3ce-d806-4856-a5a3-12f6d069c821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.046513 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29f3ce-d806-4856-a5a3-12f6d069c821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.046748 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29f3ce-d806-4856-a5a3-12f6d069c821 admin admin] PATH_INFO: `/v3/users/9369796f4c5b41d6b9fe1e786fbd4b38` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.048459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29f3ce-d806-4856-a5a3-12f6d069c821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9369796f4c5b41d6b9fe1e786fbd4b38)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:11.050431 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29f3ce-d806-4856-a5a3-12f6d069c821 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:11.055055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1059/2137] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:40:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:11.067523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7145d74-7139-46eb-9caa-774f49f5ccf8 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.067809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7145d74-7139-46eb-9caa-774f49f5ccf8 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.068085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7145d74-7139-46eb-9caa-774f49f5ccf8 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.070431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1060/2138] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 08:40:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 08:40:11.076942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf0aa9-bb16-4d69-8c68-69ae9ea7b23a None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.077269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf0aa9-bb16-4d69-8c68-69ae9ea7b23a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.077554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf0aa9-bb16-4d69-8c68-69ae9ea7b23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.101693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-d1cf0aa9-bb16-4d69-8c68-69ae9ea7b23a None None] MFA Rules not processed for user `6b40ee263df74e58bf3f958c01eb742f`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:11.102934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d1cf0aa9-bb16-4d69-8c68-69ae9ea7b23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:11.113865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1061/2139] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 08:40:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:40:11.131188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf0aa9-bb16-4d69-8c68-69ae9ea7b23a None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:11.145538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-6b29f3ce-d806-4856-a5a3-12f6d069c821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9369796f4c5b41d6b9fe1e786fbd4b38'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.162780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf0aa9-bb16-4d69-8c68-69ae9ea7b23a None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:11.168034 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-6b29f3ce-d806-4856-a5a3-12f6d069c821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9369796f4c5b41d6b9fe1e786fbd4b38'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.169148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-6b29f3ce-d806-4856-a5a3-12f6d069c821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9369796f4c5b41d6b9fe1e786fbd4b38'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.176053 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f01eb-5a2f-4580-bf7d-77f5c5f4c07a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:11.176840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f01eb-5a2f-4580-bf7d-77f5c5f4c07a service placement] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.177661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f01eb-5a2f-4580-bf7d-77f5c5f4c07a service placement] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.177661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f01eb-5a2f-4580-bf7d-77f5c5f4c07a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.180079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d41f01eb-5a2f-4580-bf7d-77f5c5f4c07a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:11.181520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-6b29f3ce-d806-4856-a5a3-12f6d069c821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9369796f4c5b41d6b9fe1e786fbd4b38'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.182436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-6b29f3ce-d806-4856-a5a3-12f6d069c821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9369796f4c5b41d6b9fe1e786fbd4b38'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.196138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f01eb-5a2f-4580-bf7d-77f5c5f4c07a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:11.205972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f01eb-5a2f-4580-bf7d-77f5c5f4c07a service placement] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:11.209907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1062/2140] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:40:11.222381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1079/2141] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:40:11 2022] DELETE /identity/v3/users/9369796f4c5b41d6b9fe1e786fbd4b38 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:40:11.223240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f01eb-5a2f-4580-bf7d-77f5c5f4c07a service placement] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:11.225885 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d41f01eb-5a2f-4580-bf7d-77f5c5f4c07a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:11.262002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f01eb-5a2f-4580-bf7d-77f5c5f4c07a service placement] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:11.337236 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:11.338723 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.339423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.339894 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] PATH_INFO: `/v3/projects/cd5d9c471dfd4410925e45a1b2585e0b` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.341639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd5d9c471dfd4410925e45a1b2585e0b)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:11.344654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:11.400791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:11.410058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd5d9c471dfd4410925e45a1b2585e0b', 'request_id': 'req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.417837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd5d9c471dfd4410925e45a1b2585e0b', 'request_id': 'req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.429167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd5d9c471dfd4410925e45a1b2585e0b', 'request_id': 'req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.443904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] The token cache is being invalidate because project cd5d9c471dfd4410925e45a1b2585e0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:40:11.444767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.445540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.447043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1063/2142] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:40:11 2022] DELETE /identity/v3/projects/cd5d9c471dfd4410925e45a1b2585e0b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:11.452138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f0da1-10b7-4153-b94f-2a9893a19075 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.452501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f0da1-10b7-4153-b94f-2a9893a19075 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.452845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f0da1-10b7-4153-b94f-2a9893a19075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.464429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:11.467553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:11.475249 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-247f0da1-10b7-4153-b94f-2a9893a19075 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:11.476203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-247f0da1-10b7-4153-b94f-2a9893a19075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:11.542487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1080/2143] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:11.550917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a38cc-6e83-491e-b44b-ce041b4ea340 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.551269 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a38cc-6e83-491e-b44b-ce041b4ea340 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.551618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a38cc-6e83-491e-b44b-ce041b4ea340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.557243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cd996-969a-4b0d-aed8-dd6c53fefb7c admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:11.562149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ea943-75cf-454e-9309-27822b9a5d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:11.562963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ea943-75cf-454e-9309-27822b9a5d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.563245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ea943-75cf-454e-9309-27822b9a5d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.563521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ea943-75cf-454e-9309-27822b9a5d6f admin admin] PATH_INFO: `/v3/projects/dc7abcbb99b7466fb1581e1010b4a7e1` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.564679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ea943-75cf-454e-9309-27822b9a5d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc7abcbb99b7466fb1581e1010b4a7e1)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:11.566631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ea943-75cf-454e-9309-27822b9a5d6f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:11.571036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-332a38cc-6e83-491e-b44b-ce041b4ea340 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:11.572256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-332a38cc-6e83-491e-b44b-ce041b4ea340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:11.581875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1081/2144] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:40:11.586111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a38cc-6e83-491e-b44b-ce041b4ea340 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:11.596686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a38cc-6e83-491e-b44b-ce041b4ea340 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:11.601589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8999e60-97e8-46c3-a337-3935726697e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:11.602201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8999e60-97e8-46c3-a337-3935726697e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.602481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8999e60-97e8-46c3-a337-3935726697e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.602754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8999e60-97e8-46c3-a337-3935726697e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.603401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8999e60-97e8-46c3-a337-3935726697e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:40:11.603706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8999e60-97e8-46c3-a337-3935726697e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:11.605342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8999e60-97e8-46c3-a337-3935726697e5 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:11.617768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-469ea943-75cf-454e-9309-27822b9a5d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc7abcbb99b7466fb1581e1010b4a7e1', 'request_id': 'req-469ea943-75cf-454e-9309-27822b9a5d6f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.618285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1082/2145] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:40:11.622160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8999e60-97e8-46c3-a337-3935726697e5 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:11.626362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-469ea943-75cf-454e-9309-27822b9a5d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc7abcbb99b7466fb1581e1010b4a7e1', 'request_id': 'req-469ea943-75cf-454e-9309-27822b9a5d6f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.631907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8999e60-97e8-46c3-a337-3935726697e5 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:11.634190 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-469ea943-75cf-454e-9309-27822b9a5d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc7abcbb99b7466fb1581e1010b4a7e1', 'request_id': 'req-469ea943-75cf-454e-9309-27822b9a5d6f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.647299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-469ea943-75cf-454e-9309-27822b9a5d6f admin admin] The token cache is being invalidate because project dc7abcbb99b7466fb1581e1010b4a7e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:40:11.647587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-469ea943-75cf-454e-9309-27822b9a5d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.648260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-469ea943-75cf-454e-9309-27822b9a5d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:11.649810 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1064/2146] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:40:11 2022] DELETE /identity/v3/projects/dc7abcbb99b7466fb1581e1010b4a7e1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:11.651329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:11.655424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056ca1c-e009-4775-8520-5213bec5a432 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.655704 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056ca1c-e009-4775-8520-5213bec5a432 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.655974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056ca1c-e009-4775-8520-5213bec5a432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.678532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-4056ca1c-e009-4775-8520-5213bec5a432 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:11.679753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4056ca1c-e009-4775-8520-5213bec5a432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:11.697808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c86a5-fe49-4b75-86e6-42bb4e746cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:11.698393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c86a5-fe49-4b75-86e6-42bb4e746cbb admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.698679 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c86a5-fe49-4b75-86e6-42bb4e746cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.698957 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c86a5-fe49-4b75-86e6-42bb4e746cbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.699637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c86a5-fe49-4b75-86e6-42bb4e746cbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:11.701559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c86a5-fe49-4b75-86e6-42bb4e746cbb admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:11.736582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1083/2147] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:40:11 2022] POST /identity/v3/projects => generated 359 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:40:11.737139 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:11.738195 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c86a5-fe49-4b75-86e6-42bb4e746cbb admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:11.740578 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2c5c86a5-fe49-4b75-86e6-42bb4e746cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:11.747780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1065/2148] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:11.756791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940e6ee-8c21-4442-b473-bbba9c83c2cd None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.757152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940e6ee-8c21-4442-b473-bbba9c83c2cd None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.757474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940e6ee-8c21-4442-b473-bbba9c83c2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.780248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-a940e6ee-8c21-4442-b473-bbba9c83c2cd None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:11.782106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a940e6ee-8c21-4442-b473-bbba9c83c2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:11.823090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c86a5-fe49-4b75-86e6-42bb4e746cbb admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:11.829158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de57b6-3c76-4677-aea4-7bea12023958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:11.829746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de57b6-3c76-4677-aea4-7bea12023958 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.830030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de57b6-3c76-4677-aea4-7bea12023958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.830303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de57b6-3c76-4677-aea4-7bea12023958 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.830940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de57b6-3c76-4677-aea4-7bea12023958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:11.832857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de57b6-3c76-4677-aea4-7bea12023958 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:11.860027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1084/2149] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:40:11 2022] POST /identity/v3/users => generated 280 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:40:11.860226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1066/2150] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:11.861212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de57b6-3c76-4677-aea4-7bea12023958 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:11.862897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940e6ee-8c21-4442-b473-bbba9c83c2cd None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:11.872462 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940e6ee-8c21-4442-b473-bbba9c83c2cd None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:11.874552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de57b6-3c76-4677-aea4-7bea12023958 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:11.877284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583fa5c-e2b5-4cc7-9008-54936d792b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:11.877859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583fa5c-e2b5-4cc7-9008-54936d792b91 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.878143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583fa5c-e2b5-4cc7-9008-54936d792b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.878418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583fa5c-e2b5-4cc7-9008-54936d792b91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.879211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583fa5c-e2b5-4cc7-9008-54936d792b91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:40:11.879384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583fa5c-e2b5-4cc7-9008-54936d792b91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:11.881026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c967ae-f788-4515-a73a-4a96aecb8de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:11.881305 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583fa5c-e2b5-4cc7-9008-54936d792b91 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:11.881860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c967ae-f788-4515-a73a-4a96aecb8de0 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.882038 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c967ae-f788-4515-a73a-4a96aecb8de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.882306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c967ae-f788-4515-a73a-4a96aecb8de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.882947 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c967ae-f788-4515-a73a-4a96aecb8de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:11.884922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c967ae-f788-4515-a73a-4a96aecb8de0 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:11.895887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1067/2151] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:40:11.900224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583fa5c-e2b5-4cc7-9008-54936d792b91 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:11.900659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1085/2152] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:11.904354 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c967ae-f788-4515-a73a-4a96aecb8de0 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:11.913662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583fa5c-e2b5-4cc7-9008-54936d792b91 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:11.916298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c967ae-f788-4515-a73a-4a96aecb8de0 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:11.918470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de558f-c2c9-4f9e-965f-716101c904cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:11.919066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de558f-c2c9-4f9e-965f-716101c904cc admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.919338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de558f-c2c9-4f9e-965f-716101c904cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.919609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de558f-c2c9-4f9e-965f-716101c904cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.920264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de558f-c2c9-4f9e-965f-716101c904cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:11.921248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ecdb4-9313-4666-9976-5c1177318041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:11.921880 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ecdb4-9313-4666-9976-5c1177318041 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:11.922157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ecdb4-9313-4666-9976-5c1177318041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:11.922266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de558f-c2c9-4f9e-965f-716101c904cc admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:11.922429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ecdb4-9313-4666-9976-5c1177318041 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:11.923061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ecdb4-9313-4666-9976-5c1177318041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:11.924630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ecdb4-9313-4666-9976-5c1177318041 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:11.937457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1086/2153] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:11.941562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ecdb4-9313-4666-9976-5c1177318041 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:11.951382 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ecdb4-9313-4666-9976-5c1177318041 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:11.952526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1068/2154] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:40:11 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:40:11.957494 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de558f-c2c9-4f9e-965f-716101c904cc admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:12.010475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb7b87-0d3a-4fa1-bf76-91be38a29f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:12.011141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb7b87-0d3a-4fa1-bf76-91be38a29f1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:12.011415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb7b87-0d3a-4fa1-bf76-91be38a29f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:12.011690 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb7b87-0d3a-4fa1-bf76-91be38a29f1e admin admin] PATH_INFO: `/v3/projects/faa733a34dd940f983d7d0a9584a8a6d/users/1354fe4e6bdc474e8ed1dadc9485306c/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:12.019819 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de558f-c2c9-4f9e-965f-716101c904cc admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:12.024891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd34b22-3b74-42a6-bb5f-ab405b5826bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:12.025518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd34b22-3b74-42a6-bb5f-ab405b5826bb admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:12.025801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd34b22-3b74-42a6-bb5f-ab405b5826bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:12.026074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd34b22-3b74-42a6-bb5f-ab405b5826bb admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:12.026706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd34b22-3b74-42a6-bb5f-ab405b5826bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:12.027185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb7b87-0d3a-4fa1-bf76-91be38a29f1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa733a34dd940f983d7d0a9584a8a6d, user_id=1354fe4e6bdc474e8ed1dadc9485306c, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:12.028694 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd34b22-3b74-42a6-bb5f-ab405b5826bb admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:12.029851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb7b87-0d3a-4fa1-bf76-91be38a29f1e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:12.033648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:12.046801 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1087/2155] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:40:11 2022] PUT /identity/v3/projects/faa733a34dd940f983d7d0a9584a8a6d/users/1354fe4e6bdc474e8ed1dadc9485306c/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:40:12.053160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1069/2156] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:40:11 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:40:12.056887 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb7b87-0d3a-4fa1-bf76-91be38a29f1e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:12.057376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6953fb0-d80f-40cf-be7c-06800b13ebb9 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:12.057664 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6953fb0-d80f-40cf-be7c-06800b13ebb9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:12.057944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6953fb0-d80f-40cf-be7c-06800b13ebb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:12.086750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-e6953fb0-d80f-40cf-be7c-06800b13ebb9 None None] MFA Rules not processed for user `1354fe4e6bdc474e8ed1dadc9485306c`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:12.087974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e6953fb0-d80f-40cf-be7c-06800b13ebb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:12.118992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb7b87-0d3a-4fa1-bf76-91be38a29f1e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:12.123723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8bdcb-0cd0-4402-bdb3-59efb4205435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:12.124330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8bdcb-0cd0-4402-bdb3-59efb4205435 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:12.124612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8bdcb-0cd0-4402-bdb3-59efb4205435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:12.125439 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8bdcb-0cd0-4402-bdb3-59efb4205435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:12.126096 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8bdcb-0cd0-4402-bdb3-59efb4205435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:12.127675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8bdcb-0cd0-4402-bdb3-59efb4205435 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:12.134862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:12.138852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1088/2157] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:12.143698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8bdcb-0cd0-4402-bdb3-59efb4205435 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:12.153764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8bdcb-0cd0-4402-bdb3-59efb4205435 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:12.158373 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc105c6c-35d2-4def-b4c8-88e8189bad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:12.158549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1070/2158] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:12.158968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc105c6c-35d2-4def-b4c8-88e8189bad4c admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:12.159245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc105c6c-35d2-4def-b4c8-88e8189bad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:12.159516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc105c6c-35d2-4def-b4c8-88e8189bad4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:12.160146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc105c6c-35d2-4def-b4c8-88e8189bad4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:12.161761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc105c6c-35d2-4def-b4c8-88e8189bad4c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:12.175427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1089/2159] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:40:12.176789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6953fb0-d80f-40cf-be7c-06800b13ebb9 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:12.190640 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6953fb0-d80f-40cf-be7c-06800b13ebb9 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:12.193456 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc105c6c-35d2-4def-b4c8-88e8189bad4c admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:12.195614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-cc105c6c-35d2-4def-b4c8-88e8189bad4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:12.196109 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ec2fb-af9d-4d42-ae33-983419c982da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:12.196867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ec2fb-af9d-4d42-ae33-983419c982da admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:12.197291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ec2fb-af9d-4d42-ae33-983419c982da admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:12.197570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ec2fb-af9d-4d42-ae33-983419c982da admin admin] PATH_INFO: `/v3/projects/c067f1df3c7f47299bc2db03d904d156/users/b0cbf49cfca74838bee4c8162dfe27ef/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:12.214988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ec2fb-af9d-4d42-ae33-983419c982da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c067f1df3c7f47299bc2db03d904d156, user_id=b0cbf49cfca74838bee4c8162dfe27ef, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:12.218389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ec2fb-af9d-4d42-ae33-983419c982da admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:12.242031 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1071/2160] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:40:12 2022] PUT /identity/v3/projects/c067f1df3c7f47299bc2db03d904d156/users/b0cbf49cfca74838bee4c8162dfe27ef/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:12.247113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395cf2b-2c1e-4654-9e5e-d6295bf4531a None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:12.247442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395cf2b-2c1e-4654-9e5e-d6295bf4531a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:12.247753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395cf2b-2c1e-4654-9e5e-d6295bf4531a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:12.263575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc105c6c-35d2-4def-b4c8-88e8189bad4c admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:12.278296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-a395cf2b-2c1e-4654-9e5e-d6295bf4531a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:12.279554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a395cf2b-2c1e-4654-9e5e-d6295bf4531a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:12.326596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ba13d-46c4-4e77-82f8-f07ea8e1a37f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:12.327217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ba13d-46c4-4e77-82f8-f07ea8e1a37f service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:12.327498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ba13d-46c4-4e77-82f8-f07ea8e1a37f service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:12.327799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ba13d-46c4-4e77-82f8-f07ea8e1a37f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:12.350534 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1072/2161] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:40:12.360001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395cf2b-2c1e-4654-9e5e-d6295bf4531a None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:12.361565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a395cf2b-2c1e-4654-9e5e-d6295bf4531a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:12.387237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:12.398184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ba13d-46c4-4e77-82f8-f07ea8e1a37f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:12.400221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ba13d-46c4-4e77-82f8-f07ea8e1a37f service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:12.411514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1090/2162] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:40:12.431598 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ba13d-46c4-4e77-82f8-f07ea8e1a37f service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:12.435546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395cf2b-2c1e-4654-9e5e-d6295bf4531a None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:12.441853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d9960-3627-45cc-9e1a-3b5d7e98c734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:12.442597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d9960-3627-45cc-9e1a-3b5d7e98c734 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:12.443444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d9960-3627-45cc-9e1a-3b5d7e98c734 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:12.443782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d9960-3627-45cc-9e1a-3b5d7e98c734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:12.444167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ba13d-46c4-4e77-82f8-f07ea8e1a37f service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:12.449072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:12.450738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:12.450885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:12.451181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] PATH_INFO: `/v3/users/1354fe4e6bdc474e8ed1dadc9485306c` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:12.453105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1354fe4e6bdc474e8ed1dadc9485306c)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:12.454452 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d9960-3627-45cc-9e1a-3b5d7e98c734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:12.455639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:12.456016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d9960-3627-45cc-9e1a-3b5d7e98c734 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:12.481560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1073/2163] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:40:12.505937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1354fe4e6bdc474e8ed1dadc9485306c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:12.529755 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1354fe4e6bdc474e8ed1dadc9485306c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:12.530309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1354fe4e6bdc474e8ed1dadc9485306c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:12.538740 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1354fe4e6bdc474e8ed1dadc9485306c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:12.539478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1354fe4e6bdc474e8ed1dadc9485306c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:12.554318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1091/2164] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:40:12 2022] DELETE /identity/v3/users/1354fe4e6bdc474e8ed1dadc9485306c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:40:12.559141 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc8b07-679f-40fc-b92d-d61ff040c9f5 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:12.559480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc8b07-679f-40fc-b92d-d61ff040c9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:12.559767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc8b07-679f-40fc-b92d-d61ff040c9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:12.581454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-a1bc8b07-679f-40fc-b92d-d61ff040c9f5 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:12.582696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a1bc8b07-679f-40fc-b92d-d61ff040c9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:12.647898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1074/2165] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:12.667085 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:12.668560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:12.731253 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23416a-d1f9-44d8-b3dc-4fd38a783e7f admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:12.735798 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f239806-8b09-43dd-a11b-22205c9b1a4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:12.736346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f239806-8b09-43dd-a11b-22205c9b1a4a service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:12.736625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f239806-8b09-43dd-a11b-22205c9b1a4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:12.736896 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f239806-8b09-43dd-a11b-22205c9b1a4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:12.746562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f239806-8b09-43dd-a11b-22205c9b1a4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:12.747998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f239806-8b09-43dd-a11b-22205c9b1a4a service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:12.758921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1092/2166] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:40:13.122920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc8b07-679f-40fc-b92d-d61ff040c9f5 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:13.124671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a1bc8b07-679f-40fc-b92d-d61ff040c9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:13.135049 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:13.143743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc8b07-679f-40fc-b92d-d61ff040c9f5 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:13.153427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:13.154305 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:13.154743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:13.155294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57 admin admin] PATH_INFO: `/v3/projects/faa733a34dd940f983d7d0a9584a8a6d` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:13.159581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa733a34dd940f983d7d0a9584a8a6d)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:13.165156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:13.234066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa733a34dd940f983d7d0a9584a8a6d', 'request_id': 'req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:13.242651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa733a34dd940f983d7d0a9584a8a6d', 'request_id': 'req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:13.252539 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa733a34dd940f983d7d0a9584a8a6d', 'request_id': 'req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:13.264977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57 admin admin] The token cache is being invalidate because project faa733a34dd940f983d7d0a9584a8a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:40:13.265300 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:13.267017 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1e0f046d-8aa7-4bc0-a16c-884d15d4be57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:13.269114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1075/2167] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:40:13 2022] DELETE /identity/v3/projects/faa733a34dd940f983d7d0a9584a8a6d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:13.273950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b6f5-374f-48f5-9e18-a80dbca69f48 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:13.274420 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b6f5-374f-48f5-9e18-a80dbca69f48 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:13.274420 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b6f5-374f-48f5-9e18-a80dbca69f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:13.301074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-91c6b6f5-374f-48f5-9e18-a80dbca69f48 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:13.302323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-91c6b6f5-374f-48f5-9e18-a80dbca69f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:13.377560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1093/2168] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:40:13.385137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb522cf-fbc9-413f-83eb-9c6c8b9c4630 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:13.385645 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb522cf-fbc9-413f-83eb-9c6c8b9c4630 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:13.385814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb522cf-fbc9-413f-83eb-9c6c8b9c4630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:13.407745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-edb522cf-fbc9-413f-83eb-9c6c8b9c4630 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:13.409072 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-edb522cf-fbc9-413f-83eb-9c6c8b9c4630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:13.417828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1076/2169] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:13.422173 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6b6f5-374f-48f5-9e18-a80dbca69f48 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:13.432564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6b6f5-374f-48f5-9e18-a80dbca69f48 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:13.438453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d6d84-107b-40f4-8f2d-dc4f0cbfdbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:13.439728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d6d84-107b-40f4-8f2d-dc4f0cbfdbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:13.439728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d6d84-107b-40f4-8f2d-dc4f0cbfdbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:13.439728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d6d84-107b-40f4-8f2d-dc4f0cbfdbe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:13.440493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d6d84-107b-40f4-8f2d-dc4f0cbfdbe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:40:13.440870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d6d84-107b-40f4-8f2d-dc4f0cbfdbe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:13.442648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d6d84-107b-40f4-8f2d-dc4f0cbfdbe0 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:13.444363 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:13.462511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1094/2170] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:40:13.463397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb522cf-fbc9-413f-83eb-9c6c8b9c4630 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:13.473907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb522cf-fbc9-413f-83eb-9c6c8b9c4630 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:13.479014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82cbae-0e6c-459e-b27a-91594a86b6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:13.479622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82cbae-0e6c-459e-b27a-91594a86b6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:13.479897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82cbae-0e6c-459e-b27a-91594a86b6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:13.480169 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82cbae-0e6c-459e-b27a-91594a86b6f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:13.480803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82cbae-0e6c-459e-b27a-91594a86b6f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:13.482905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82cbae-0e6c-459e-b27a-91594a86b6f4 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:13.529773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1077/2171] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:40:13 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:40:13.534723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d6d84-107b-40f4-8f2d-dc4f0cbfdbe0 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:13.545874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d6d84-107b-40f4-8f2d-dc4f0cbfdbe0 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:13.614612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36b555-d4a5-42b0-8976-3c71e7a6fa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:13.615306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36b555-d4a5-42b0-8976-3c71e7a6fa11 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:13.615592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36b555-d4a5-42b0-8976-3c71e7a6fa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:13.616045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36b555-d4a5-42b0-8976-3c71e7a6fa11 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:13.616717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36b555-d4a5-42b0-8976-3c71e7a6fa11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:13.618874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36b555-d4a5-42b0-8976-3c71e7a6fa11 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:13.660056 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1095/2172] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:40:13 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:40:13.664238 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82cbae-0e6c-459e-b27a-91594a86b6f4 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:13.677640 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82cbae-0e6c-459e-b27a-91594a86b6f4 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:13.688827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640f75c-9e75-47ff-bfbe-c58a98d0bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:13.689653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640f75c-9e75-47ff-bfbe-c58a98d0bab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:13.689948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640f75c-9e75-47ff-bfbe-c58a98d0bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:13.690397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640f75c-9e75-47ff-bfbe-c58a98d0bab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:13.691069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640f75c-9e75-47ff-bfbe-c58a98d0bab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:13.692699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640f75c-9e75-47ff-bfbe-c58a98d0bab2 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:13.708317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1078/2173] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:13.709667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36b555-d4a5-42b0-8976-3c71e7a6fa11 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:13.722415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36b555-d4a5-42b0-8976-3c71e7a6fa11 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:13.733009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc113d64-0f4b-4f16-a615-02cd958396a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:13.733724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc113d64-0f4b-4f16-a615-02cd958396a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:13.734015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc113d64-0f4b-4f16-a615-02cd958396a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:13.734330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc113d64-0f4b-4f16-a615-02cd958396a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:13.735039 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc113d64-0f4b-4f16-a615-02cd958396a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:13.736755 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc113d64-0f4b-4f16-a615-02cd958396a0 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:13.751584 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1096/2174] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:40:13.755962 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640f75c-9e75-47ff-bfbe-c58a98d0bab2 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:13.770386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640f75c-9e75-47ff-bfbe-c58a98d0bab2 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:13.775989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b95a65-0366-420d-9332-35e65db0a97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:13.776726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b95a65-0366-420d-9332-35e65db0a97c admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:13.777011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b95a65-0366-420d-9332-35e65db0a97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:13.777333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b95a65-0366-420d-9332-35e65db0a97c admin admin] PATH_INFO: `/v3/projects/6fe65abbf7124e6b90611bef9a30d8ff/users/5cd56771bf5d4aaa8471f3093a1ef6e1/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:13.795628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b95a65-0366-420d-9332-35e65db0a97c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe65abbf7124e6b90611bef9a30d8ff, user_id=5cd56771bf5d4aaa8471f3093a1ef6e1, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:13.799052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b95a65-0366-420d-9332-35e65db0a97c admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:13.821753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1079/2175] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:40:13 2022] PUT /identity/v3/projects/6fe65abbf7124e6b90611bef9a30d8ff/users/5cd56771bf5d4aaa8471f3093a1ef6e1/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:40:13.833211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1064a-7d90-4556-8213-048c82a09a07 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:13.833501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1064a-7d90-4556-8213-048c82a09a07 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:13.833780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1064a-7d90-4556-8213-048c82a09a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:13.863780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-a1a1064a-7d90-4556-8213-048c82a09a07 None None] MFA Rules not processed for user `5cd56771bf5d4aaa8471f3093a1ef6e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:13.865281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a1a1064a-7d90-4556-8213-048c82a09a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:13.957000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1097/2176] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:13.958113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b95a65-0366-420d-9332-35e65db0a97c admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:13.960803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-35b95a65-0366-420d-9332-35e65db0a97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:14.031940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b95a65-0366-420d-9332-35e65db0a97c admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:14.036061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef76fe7-ceb5-44ce-837c-f542c473c06e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:14.036548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef76fe7-ceb5-44ce-837c-f542c473c06e service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:14.036789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef76fe7-ceb5-44ce-837c-f542c473c06e service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:14.037126 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef76fe7-ceb5-44ce-837c-f542c473c06e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:14.047337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef76fe7-ceb5-44ce-837c-f542c473c06e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:14.049067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef76fe7-ceb5-44ce-837c-f542c473c06e service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:14.059162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1080/2177] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:40:14.089644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1064a-7d90-4556-8213-048c82a09a07 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:14.219622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1064a-7d90-4556-8213-048c82a09a07 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:14.224462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:14.225219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:14.225575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:14.225926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] PATH_INFO: `/v3/users/5cd56771bf5d4aaa8471f3093a1ef6e1` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:14.227181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cd56771bf5d4aaa8471f3093a1ef6e1)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:14.229288 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:14.232141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:14.288365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cd56771bf5d4aaa8471f3093a1ef6e1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:14.303176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cd56771bf5d4aaa8471f3093a1ef6e1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:14.303837 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cd56771bf5d4aaa8471f3093a1ef6e1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:14.311370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cd56771bf5d4aaa8471f3093a1ef6e1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:14.312061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cd56771bf5d4aaa8471f3093a1ef6e1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:14.327205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1098/2178] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:40:14 2022] DELETE /identity/v3/users/5cd56771bf5d4aaa8471f3093a1ef6e1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:40:14.332103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e8894-1c96-4e8f-992a-bd94616c6857 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:14.332659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e8894-1c96-4e8f-992a-bd94616c6857 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:14.332997 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e8894-1c96-4e8f-992a-bd94616c6857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:14.354836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-360e8894-1c96-4e8f-992a-bd94616c6857 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:14.356386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-360e8894-1c96-4e8f-992a-bd94616c6857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:14.385274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:14.435723 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1081/2179] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:14.445476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:14.448304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:14.523822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9dd26-0aee-47db-912b-e5df6afc484a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:14.529731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b021e4-142e-4867-8963-89bcd0942bc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:14.530423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b021e4-142e-4867-8963-89bcd0942bc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:14.530765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b021e4-142e-4867-8963-89bcd0942bc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:14.531783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b021e4-142e-4867-8963-89bcd0942bc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:14.541813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b021e4-142e-4867-8963-89bcd0942bc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:14.543254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b021e4-142e-4867-8963-89bcd0942bc4 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:14.554086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1099/2180] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:40:14.887202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e8894-1c96-4e8f-992a-bd94616c6857 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:14.888678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-360e8894-1c96-4e8f-992a-bd94616c6857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:14.918157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e8894-1c96-4e8f-992a-bd94616c6857 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:14.927922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:14.928597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:14.928884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:14.929196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0 admin admin] PATH_INFO: `/v3/projects/6fe65abbf7124e6b90611bef9a30d8ff` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:14.931183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fe65abbf7124e6b90611bef9a30d8ff)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:14.935264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:15.024548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fe65abbf7124e6b90611bef9a30d8ff', 'request_id': 'req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:15.032355 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fe65abbf7124e6b90611bef9a30d8ff', 'request_id': 'req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:15.063865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fe65abbf7124e6b90611bef9a30d8ff', 'request_id': 'req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:15.088444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0 admin admin] The token cache is being invalidate because project 6fe65abbf7124e6b90611bef9a30d8ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:40:15.088742 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:15.089446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3293856a-aa5d-40ef-8c3f-6d0b9b4bf0d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:15.091144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1082/2181] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:40:14 2022] DELETE /identity/v3/projects/6fe65abbf7124e6b90611bef9a30d8ff => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:15.096525 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefc3a1-9ce7-470d-8659-6180b010b7f9 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:15.097021 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefc3a1-9ce7-470d-8659-6180b010b7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:15.097831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefc3a1-9ce7-470d-8659-6180b010b7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:15.129167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-beefc3a1-9ce7-470d-8659-6180b010b7f9 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:15.131408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-beefc3a1-9ce7-470d-8659-6180b010b7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:15.236463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1100/2182] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:15.244658 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea56f62-8908-45f1-91a0-5021476f4771 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:15.244946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea56f62-8908-45f1-91a0-5021476f4771 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:15.245302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea56f62-8908-45f1-91a0-5021476f4771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:15.272655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-0ea56f62-8908-45f1-91a0-5021476f4771 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:15.274000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-0ea56f62-8908-45f1-91a0-5021476f4771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:15.287294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1083/2183] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:15.288281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefc3a1-9ce7-470d-8659-6180b010b7f9 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:15.299227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefc3a1-9ce7-470d-8659-6180b010b7f9 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:15.304190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec134414-456d-44aa-94ab-30ff91a718ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:15.304870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec134414-456d-44aa-94ab-30ff91a718ed admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:15.305263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec134414-456d-44aa-94ab-30ff91a718ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:15.305625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec134414-456d-44aa-94ab-30ff91a718ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:15.306625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec134414-456d-44aa-94ab-30ff91a718ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:40:15.306998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec134414-456d-44aa-94ab-30ff91a718ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:15.308667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec134414-456d-44aa-94ab-30ff91a718ed admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:15.340219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1101/2184] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:40:15.340744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea56f62-8908-45f1-91a0-5021476f4771 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:15.353189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea56f62-8908-45f1-91a0-5021476f4771 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:15.367355 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6ef61-05e1-41c1-b580-7ed295721876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:15.369242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6ef61-05e1-41c1-b580-7ed295721876 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:15.369537 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6ef61-05e1-41c1-b580-7ed295721876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:15.369807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6ef61-05e1-41c1-b580-7ed295721876 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:15.373158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6ef61-05e1-41c1-b580-7ed295721876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:15.378604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6ef61-05e1-41c1-b580-7ed295721876 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:15.433308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1084/2185] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:40:15 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:40:15.434880 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec134414-456d-44aa-94ab-30ff91a718ed admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:15.447603 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec134414-456d-44aa-94ab-30ff91a718ed admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:15.468183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:15.561533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649769e-0ff9-4974-965d-60fc35b51c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:15.562971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649769e-0ff9-4974-965d-60fc35b51c67 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:15.563480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649769e-0ff9-4974-965d-60fc35b51c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:15.565645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649769e-0ff9-4974-965d-60fc35b51c67 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:15.566419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649769e-0ff9-4974-965d-60fc35b51c67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:15.568659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649769e-0ff9-4974-965d-60fc35b51c67 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:15.605071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6ef61-05e1-41c1-b580-7ed295721876 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:15.607356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1102/2186] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:40:15 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:40:15.625721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6ef61-05e1-41c1-b580-7ed295721876 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:15.638070 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727768dc-ea87-4220-b6cc-831179f4ccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:15.639423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727768dc-ea87-4220-b6cc-831179f4ccc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:15.639958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727768dc-ea87-4220-b6cc-831179f4ccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:15.643121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727768dc-ea87-4220-b6cc-831179f4ccc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:15.643121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727768dc-ea87-4220-b6cc-831179f4ccc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:15.646345 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727768dc-ea87-4220-b6cc-831179f4ccc2 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:15.667318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1085/2187] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:15.671380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649769e-0ff9-4974-965d-60fc35b51c67 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:15.685167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649769e-0ff9-4974-965d-60fc35b51c67 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:15.693512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aca3c0-0f6f-4365-8f6d-4e5b99380ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:15.694523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aca3c0-0f6f-4365-8f6d-4e5b99380ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:15.695026 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aca3c0-0f6f-4365-8f6d-4e5b99380ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:15.695498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aca3c0-0f6f-4365-8f6d-4e5b99380ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:15.696696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aca3c0-0f6f-4365-8f6d-4e5b99380ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:15.698848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aca3c0-0f6f-4365-8f6d-4e5b99380ba1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:15.716408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1103/2188] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:15.718633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727768dc-ea87-4220-b6cc-831179f4ccc2 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:15.731948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727768dc-ea87-4220-b6cc-831179f4ccc2 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:15.741245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276dd62-9494-4a4b-90e7-898009ebb4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:15.741981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276dd62-9494-4a4b-90e7-898009ebb4d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:15.742361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276dd62-9494-4a4b-90e7-898009ebb4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:15.742555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276dd62-9494-4a4b-90e7-898009ebb4d0 admin admin] PATH_INFO: `/v3/projects/c507f95ba3c34cc789ee8903b40c346d/users/4f250207e16c4515bd9beb9ca4b1fde6/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:15.761236 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276dd62-9494-4a4b-90e7-898009ebb4d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c507f95ba3c34cc789ee8903b40c346d, user_id=4f250207e16c4515bd9beb9ca4b1fde6, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:15.764020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276dd62-9494-4a4b-90e7-898009ebb4d0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:15.780446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1086/2189] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:40:15 2022] PUT /identity/v3/projects/c507f95ba3c34cc789ee8903b40c346d/users/4f250207e16c4515bd9beb9ca4b1fde6/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:15.785877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5892be-5c22-459f-8952-e8c816d0dbe2 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:15.786074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5892be-5c22-459f-8952-e8c816d0dbe2 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:15.786334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5892be-5c22-459f-8952-e8c816d0dbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:15.808018 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5d5892be-5c22-459f-8952-e8c816d0dbe2 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:15.809565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5d5892be-5c22-459f-8952-e8c816d0dbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:15.881766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1104/2190] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:15.896608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276dd62-9494-4a4b-90e7-898009ebb4d0 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:15.898489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3276dd62-9494-4a4b-90e7-898009ebb4d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:15.968281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:15.975948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276dd62-9494-4a4b-90e7-898009ebb4d0 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:15.981443 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe97f5-15f4-4f58-8f87-40cd2c3a306b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:15.982586 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe97f5-15f4-4f58-8f87-40cd2c3a306b service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:15.983060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe97f5-15f4-4f58-8f87-40cd2c3a306b service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:15.983614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe97f5-15f4-4f58-8f87-40cd2c3a306b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:15.995414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe97f5-15f4-4f58-8f87-40cd2c3a306b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:15.997000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe97f5-15f4-4f58-8f87-40cd2c3a306b service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:16.008437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1087/2191] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:40:17.855021 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d575501-de25-4a37-b278-3471f4f5e4e3 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:17.855592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d575501-de25-4a37-b278-3471f4f5e4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:17.856056 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d575501-de25-4a37-b278-3471f4f5e4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:17.910551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-3d575501-de25-4a37-b278-3471f4f5e4e3 None None] MFA Rules not processed for user `b0cbf49cfca74838bee4c8162dfe27ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:17.912644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3d575501-de25-4a37-b278-3471f4f5e4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:18.007465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1105/2192] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:40:18.022717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe97f5-15f4-4f58-8f87-40cd2c3a306b service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:18.024511 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1fbe97f5-15f4-4f58-8f87-40cd2c3a306b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:18.145396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe97f5-15f4-4f58-8f87-40cd2c3a306b service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:18.152407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4efc-3ab1-40d0-91a9-33b8c961acae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:18.153397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4efc-3ab1-40d0-91a9-33b8c961acae service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:18.153777 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4efc-3ab1-40d0-91a9-33b8c961acae service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:18.154283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4efc-3ab1-40d0-91a9-33b8c961acae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:18.165218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c4efc-3ab1-40d0-91a9-33b8c961acae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:18.167418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c4efc-3ab1-40d0-91a9-33b8c961acae service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:18.178915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1088/2193] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:40:20.316294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d862f-2bb3-469a-a717-f84ca52ed6b0 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:20.316753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d862f-2bb3-469a-a717-f84ca52ed6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:20.316854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d862f-2bb3-469a-a717-f84ca52ed6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:20.359158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-d69d862f-2bb3-469a-a717-f84ca52ed6b0 None None] MFA Rules not processed for user `b0cbf49cfca74838bee4c8162dfe27ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:20.367564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d69d862f-2bb3-469a-a717-f84ca52ed6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:20.395085 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1106/2194] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:24.531231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453a967-30fe-4a6a-b3d5-8319d0f2c6bd None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:24.531718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453a967-30fe-4a6a-b3d5-8319d0f2c6bd None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:24.531930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453a967-30fe-4a6a-b3d5-8319d0f2c6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:24.621785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-2453a967-30fe-4a6a-b3d5-8319d0f2c6bd None None] MFA Rules not processed for user `4f250207e16c4515bd9beb9ca4b1fde6`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:24.623340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2453a967-30fe-4a6a-b3d5-8319d0f2c6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:24.813233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1089/2195] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:24.817293 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d862f-2bb3-469a-a717-f84ca52ed6b0 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:24.836121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d862f-2bb3-469a-a717-f84ca52ed6b0 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:24.846206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c81eba-618e-456a-8ce5-73ad9e6cd150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:24.847090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c81eba-618e-456a-8ce5-73ad9e6cd150 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:24.847410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c81eba-618e-456a-8ce5-73ad9e6cd150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:24.847632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c81eba-618e-456a-8ce5-73ad9e6cd150 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:24.848298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c81eba-618e-456a-8ce5-73ad9e6cd150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:24.850198 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c81eba-618e-456a-8ce5-73ad9e6cd150 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:24.875895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:24.917039 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1107/2196] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:40:24 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:40:24.922885 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453a967-30fe-4a6a-b3d5-8319d0f2c6bd None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:24.942521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453a967-30fe-4a6a-b3d5-8319d0f2c6bd None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:25.053428 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d3b7f-d51f-49d0-8a02-9c973a5ac0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:25.054231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d3b7f-d51f-49d0-8a02-9c973a5ac0ff admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:25.054983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d3b7f-d51f-49d0-8a02-9c973a5ac0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:25.055260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d3b7f-d51f-49d0-8a02-9c973a5ac0ff admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:25.056615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d3b7f-d51f-49d0-8a02-9c973a5ac0ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:25.061884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d3b7f-d51f-49d0-8a02-9c973a5ac0ff admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:25.079857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:25.115821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1090/2197] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:40:24 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:40:25.117417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c81eba-618e-456a-8ce5-73ad9e6cd150 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:25.132096 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c81eba-618e-456a-8ce5-73ad9e6cd150 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:25.145750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf378b-3902-43b1-8807-ebb69369b5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:25.147519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf378b-3902-43b1-8807-ebb69369b5cb admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:25.148368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf378b-3902-43b1-8807-ebb69369b5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:25.150430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf378b-3902-43b1-8807-ebb69369b5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:25.151346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf378b-3902-43b1-8807-ebb69369b5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:25.154809 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf378b-3902-43b1-8807-ebb69369b5cb admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:25.203772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1108/2198] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:25.205535 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d3b7f-d51f-49d0-8a02-9c973a5ac0ff admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:25.222943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d3b7f-d51f-49d0-8a02-9c973a5ac0ff admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:25.227888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ee3c0-e8b2-45be-9655-85cce3ff5dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:25.228565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ee3c0-e8b2-45be-9655-85cce3ff5dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:25.228845 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ee3c0-e8b2-45be-9655-85cce3ff5dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:25.229338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ee3c0-e8b2-45be-9655-85cce3ff5dcc admin admin] PATH_INFO: `/v3/projects/f6dac82b091e40b38f9fd37de64e1566/users/ab78cd046b2742de85d5d11501b78572/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:25.247864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ee3c0-e8b2-45be-9655-85cce3ff5dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6dac82b091e40b38f9fd37de64e1566, user_id=ab78cd046b2742de85d5d11501b78572, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:25.253887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ee3c0-e8b2-45be-9655-85cce3ff5dcc admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:25.291160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1091/2199] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:40:25 2022] PUT /identity/v3/projects/f6dac82b091e40b38f9fd37de64e1566/users/ab78cd046b2742de85d5d11501b78572/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:25.292872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf378b-3902-43b1-8807-ebb69369b5cb admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:25.344974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf378b-3902-43b1-8807-ebb69369b5cb admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:25.542928 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db3c17-327d-49fc-8016-8a28ce37d48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:25.544205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db3c17-327d-49fc-8016-8a28ce37d48c admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:25.544778 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db3c17-327d-49fc-8016-8a28ce37d48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:25.545412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db3c17-327d-49fc-8016-8a28ce37d48c admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:25.546534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db3c17-327d-49fc-8016-8a28ce37d48c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:25.548486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db3c17-327d-49fc-8016-8a28ce37d48c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:25.567333 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1109/2200] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:25.572470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ee3c0-e8b2-45be-9655-85cce3ff5dcc admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:25.590697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ee3c0-e8b2-45be-9655-85cce3ff5dcc admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:25.611739 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2fc23-f7f4-4499-9c46-0aa1bc986c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:25.613077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2fc23-f7f4-4499-9c46-0aa1bc986c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:25.615753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2fc23-f7f4-4499-9c46-0aa1bc986c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:25.616347 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2fc23-f7f4-4499-9c46-0aa1bc986c10 admin admin] PATH_INFO: `/v3/projects/f6dac82b091e40b38f9fd37de64e1566/users/ab78cd046b2742de85d5d11501b78572/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:25.619807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2fc23-f7f4-4499-9c46-0aa1bc986c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6dac82b091e40b38f9fd37de64e1566, user_id=ab78cd046b2742de85d5d11501b78572, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:25.623583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2fc23-f7f4-4499-9c46-0aa1bc986c10 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:25.645367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1092/2201] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:40:25 2022] PUT /identity/v3/projects/f6dac82b091e40b38f9fd37de64e1566/users/ab78cd046b2742de85d5d11501b78572/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:27.243715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f88eef-8b2a-43f9-9c61-eec37711918f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:27.244218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f88eef-8b2a-43f9-9c61-eec37711918f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:27.244341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f88eef-8b2a-43f9-9c61-eec37711918f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:27.286159 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e7f88eef-8b2a-43f9-9c61-eec37711918f None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:27.292050 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e7f88eef-8b2a-43f9-9c61-eec37711918f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:27.424720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1110/2202] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:40:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:27.494526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2fc23-f7f4-4499-9c46-0aa1bc986c10 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:27.497043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-45b2fc23-f7f4-4499-9c46-0aa1bc986c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:27.667454 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2fc23-f7f4-4499-9c46-0aa1bc986c10 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:27.679498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0293a-bf84-4133-892b-dccb8b6857b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:27.680224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0293a-bf84-4133-892b-dccb8b6857b8 service swift] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:27.680534 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0293a-bf84-4133-892b-dccb8b6857b8 service swift] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:27.680813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0293a-bf84-4133-892b-dccb8b6857b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:27.712296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0293a-bf84-4133-892b-dccb8b6857b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:27.716790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0293a-bf84-4133-892b-dccb8b6857b8 service swift] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:27.727500 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1093/2203] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:40:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:40:28.659183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7449066-c102-42a6-9196-909b594c366a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:28.659738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7449066-c102-42a6-9196-909b594c366a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:28.663874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7449066-c102-42a6-9196-909b594c366a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:28.699718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f7449066-c102-42a6-9196-909b594c366a None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:28.701309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f7449066-c102-42a6-9196-909b594c366a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:28.726034 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1111/2204] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 08:40:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:28.734672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0293a-bf84-4133-892b-dccb8b6857b8 service swift] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:28.759757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0293a-bf84-4133-892b-dccb8b6857b8 service swift] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:28.775222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9a2b2-df79-44e6-a717-c3b72d84a00d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:28.776282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9a2b2-df79-44e6-a717-c3b72d84a00d service swift] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:28.776729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9a2b2-df79-44e6-a717-c3b72d84a00d service swift] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:28.777388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9a2b2-df79-44e6-a717-c3b72d84a00d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:28.805662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9a2b2-df79-44e6-a717-c3b72d84a00d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:28.807709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9a2b2-df79-44e6-a717-c3b72d84a00d service swift] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:28.823156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1094/2205] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:40:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:40:29.170366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d8ba5-e543-4e4b-a22d-6c00593553e7 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:29.170601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d8ba5-e543-4e4b-a22d-6c00593553e7 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:29.170878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d8ba5-e543-4e4b-a22d-6c00593553e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:29.193362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-403d8ba5-e543-4e4b-a22d-6c00593553e7 None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:29.195895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-403d8ba5-e543-4e4b-a22d-6c00593553e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:29.207816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1112/2206] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:40:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:29.226547 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9a2b2-df79-44e6-a717-c3b72d84a00d service swift] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:29.238520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9a2b2-df79-44e6-a717-c3b72d84a00d service swift] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:29.243837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdb4c5-698c-4309-9959-f1669540b7dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:29.244674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdb4c5-698c-4309-9959-f1669540b7dd service swift] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:29.245134 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdb4c5-698c-4309-9959-f1669540b7dd service swift] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:29.245560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdb4c5-698c-4309-9959-f1669540b7dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:29.255786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acdb4c5-698c-4309-9959-f1669540b7dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:29.257441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acdb4c5-698c-4309-9959-f1669540b7dd service swift] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:29.266913 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1095/2207] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:40:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:40:29.914728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77584f76-b5aa-4726-a7c4-dbf2d594cca0 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:29.915234 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77584f76-b5aa-4726-a7c4-dbf2d594cca0 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:29.915344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77584f76-b5aa-4726-a7c4-dbf2d594cca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:29.945610 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-77584f76-b5aa-4726-a7c4-dbf2d594cca0 None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:29.947022 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-77584f76-b5aa-4726-a7c4-dbf2d594cca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:29.970642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1113/2208] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 08:40:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:29.985461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdb4c5-698c-4309-9959-f1669540b7dd service swift] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:29.998329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdb4c5-698c-4309-9959-f1669540b7dd service swift] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:30.010401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbb027-1b6b-4185-8212-17b6faa6791f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:30.011041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbb027-1b6b-4185-8212-17b6faa6791f service swift] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:30.011316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbb027-1b6b-4185-8212-17b6faa6791f service swift] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:30.011587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbb027-1b6b-4185-8212-17b6faa6791f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:30.021317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbb027-1b6b-4185-8212-17b6faa6791f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:30.022928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbb027-1b6b-4185-8212-17b6faa6791f service swift] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:30.033736 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1096/2209] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:40:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:40:32.621563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77584f76-b5aa-4726-a7c4-dbf2d594cca0 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:32.623359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-77584f76-b5aa-4726-a7c4-dbf2d594cca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:32.710198 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77584f76-b5aa-4726-a7c4-dbf2d594cca0 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:32.720241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9851e-737d-4255-8782-6a17c607f458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:32.721147 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9851e-737d-4255-8782-6a17c607f458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:32.721573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9851e-737d-4255-8782-6a17c607f458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:32.722006 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9851e-737d-4255-8782-6a17c607f458 admin admin] PATH_INFO: `/v3/users/5c45623c9421460abfa049a8040c7c12` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:32.723832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9851e-737d-4255-8782-6a17c607f458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c45623c9421460abfa049a8040c7c12)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:32.729399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9851e-737d-4255-8782-6a17c607f458 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:32.745248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:32.764613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f20f59-cfc7-4c1b-812f-937ecc1f413d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:32.765981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f20f59-cfc7-4c1b-812f-937ecc1f413d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:32.766432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f20f59-cfc7-4c1b-812f-937ecc1f413d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:32.798788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-94f20f59-cfc7-4c1b-812f-937ecc1f413d None None] MFA Rules not processed for user `ab78cd046b2742de85d5d11501b78572`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:32.800170 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-94f20f59-cfc7-4c1b-812f-937ecc1f413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:32.834544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-54b9851e-737d-4255-8782-6a17c607f458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c45623c9421460abfa049a8040c7c12'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:32.850949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-54b9851e-737d-4255-8782-6a17c607f458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c45623c9421460abfa049a8040c7c12'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:32.851693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-54b9851e-737d-4255-8782-6a17c607f458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c45623c9421460abfa049a8040c7c12'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:32.861865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-54b9851e-737d-4255-8782-6a17c607f458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c45623c9421460abfa049a8040c7c12'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:32.862627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-54b9851e-737d-4255-8782-6a17c607f458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c45623c9421460abfa049a8040c7c12'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:32.869972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:32.877559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1114/2210] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:40:32 2022] DELETE /identity/v3/users/5c45623c9421460abfa049a8040c7c12 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:32.882552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6049da2-8221-4698-9713-7e1dd28c5607 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:32.882776 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6049da2-8221-4698-9713-7e1dd28c5607 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:32.882864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6049da2-8221-4698-9713-7e1dd28c5607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:32.899415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1097/2211] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:40:32.915062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f20f59-cfc7-4c1b-812f-937ecc1f413d None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:32.917283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-94f20f59-cfc7-4c1b-812f-937ecc1f413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:32.919404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b6049da2-8221-4698-9713-7e1dd28c5607 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:32.920838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b6049da2-8221-4698-9713-7e1dd28c5607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:33.021611 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1115/2212] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:40:33.034454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6049da2-8221-4698-9713-7e1dd28c5607 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:33.036161 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b6049da2-8221-4698-9713-7e1dd28c5607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:33.045140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f20f59-cfc7-4c1b-812f-937ecc1f413d None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:33.126587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6049da2-8221-4698-9713-7e1dd28c5607 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:33.139040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac2417-5ae7-4a98-b6a7-4feb302cbf05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:33.139800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac2417-5ae7-4a98-b6a7-4feb302cbf05 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:33.140709 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac2417-5ae7-4a98-b6a7-4feb302cbf05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:33.142219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac2417-5ae7-4a98-b6a7-4feb302cbf05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:33.153339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244682a-2799-443f-8afd-c45729cc3e53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:33.154271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244682a-2799-443f-8afd-c45729cc3e53 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:33.154762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244682a-2799-443f-8afd-c45729cc3e53 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:33.154883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244682a-2799-443f-8afd-c45729cc3e53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:33.158440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1244682a-2799-443f-8afd-c45729cc3e53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:33.167299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac2417-5ae7-4a98-b6a7-4feb302cbf05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:33.168754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac2417-5ae7-4a98-b6a7-4feb302cbf05 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:33.179576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1116/2213] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:40:33.236062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244682a-2799-443f-8afd-c45729cc3e53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:33.237751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244682a-2799-443f-8afd-c45729cc3e53 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:33.248509 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1098/2214] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:40:33.646731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac2417-5ae7-4a98-b6a7-4feb302cbf05 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:33.648871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-53ac2417-5ae7-4a98-b6a7-4feb302cbf05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:33.668584 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac2417-5ae7-4a98-b6a7-4feb302cbf05 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:33.690911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16489a-a939-4e96-bd19-5bf91304d0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:33.691582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16489a-a939-4e96-bd19-5bf91304d0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:33.692495 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16489a-a939-4e96-bd19-5bf91304d0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:33.692495 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16489a-a939-4e96-bd19-5bf91304d0d9 admin admin] PATH_INFO: `/v3/projects/3777d8561b4e4c9385e87007b37fa57a` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:33.699777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16489a-a939-4e96-bd19-5bf91304d0d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3777d8561b4e4c9385e87007b37fa57a)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:33.701875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16489a-a939-4e96-bd19-5bf91304d0d9 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:33.759139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:33.806945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ee16489a-a939-4e96-bd19-5bf91304d0d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3777d8561b4e4c9385e87007b37fa57a', 'request_id': 'req-ee16489a-a939-4e96-bd19-5bf91304d0d9'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:33.814211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ee16489a-a939-4e96-bd19-5bf91304d0d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3777d8561b4e4c9385e87007b37fa57a', 'request_id': 'req-ee16489a-a939-4e96-bd19-5bf91304d0d9'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:33.822112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ee16489a-a939-4e96-bd19-5bf91304d0d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3777d8561b4e4c9385e87007b37fa57a', 'request_id': 'req-ee16489a-a939-4e96-bd19-5bf91304d0d9'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:33.856331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ee16489a-a939-4e96-bd19-5bf91304d0d9 admin admin] The token cache is being invalidate because project 3777d8561b4e4c9385e87007b37fa57a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:40:33.856683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ee16489a-a939-4e96-bd19-5bf91304d0d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:33.857550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ee16489a-a939-4e96-bd19-5bf91304d0d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:40:33.859869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1117/2215] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:40:33 2022] DELETE /identity/v3/projects/3777d8561b4e4c9385e87007b37fa57a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:33.865955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5355c2d-a287-4835-a807-3ea7cd57219f None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:33.866958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5355c2d-a287-4835-a807-3ea7cd57219f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:33.867820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5355c2d-a287-4835-a807-3ea7cd57219f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:33.894063 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-e5355c2d-a287-4835-a807-3ea7cd57219f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:33.895452 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e5355c2d-a287-4835-a807-3ea7cd57219f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:33.928673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:33.994813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1099/2216] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:34.002902 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c38789-82cc-4c71-a36e-c12eab635fa4 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:34.003215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c38789-82cc-4c71-a36e-c12eab635fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:34.003468 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c38789-82cc-4c71-a36e-c12eab635fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:34.025926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-81c38789-82cc-4c71-a36e-c12eab635fa4 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:34.027167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-81c38789-82cc-4c71-a36e-c12eab635fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:34.059905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1118/2217] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:34.062676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5355c2d-a287-4835-a807-3ea7cd57219f None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:34.084498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5355c2d-a287-4835-a807-3ea7cd57219f None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:34.090871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd858333-a2aa-4ba9-afc7-d7434fb54f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:34.091505 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd858333-a2aa-4ba9-afc7-d7434fb54f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:34.091832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd858333-a2aa-4ba9-afc7-d7434fb54f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:34.092060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd858333-a2aa-4ba9-afc7-d7434fb54f4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:34.092777 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd858333-a2aa-4ba9-afc7-d7434fb54f4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:40:34.093680 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd858333-a2aa-4ba9-afc7-d7434fb54f4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:34.096847 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd858333-a2aa-4ba9-afc7-d7434fb54f4f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:34.114066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1100/2218] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:40:34.116318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c38789-82cc-4c71-a36e-c12eab635fa4 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:34.126494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c38789-82cc-4c71-a36e-c12eab635fa4 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:34.133844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40d03b-ea92-4b15-a7da-175f7d5afa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:34.136232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40d03b-ea92-4b15-a7da-175f7d5afa97 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:34.137708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40d03b-ea92-4b15-a7da-175f7d5afa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:34.138185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40d03b-ea92-4b15-a7da-175f7d5afa97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:34.139034 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40d03b-ea92-4b15-a7da-175f7d5afa97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:34.142292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40d03b-ea92-4b15-a7da-175f7d5afa97 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:34.197601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1119/2219] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:40:34 2022] POST /identity/v3/projects => generated 391 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:40:34.202037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd858333-a2aa-4ba9-afc7-d7434fb54f4f admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:34.215079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd858333-a2aa-4ba9-afc7-d7434fb54f4f admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:34.295338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f1f4f-4a3f-4749-97c3-acf61ef2dfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:34.296211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f1f4f-4a3f-4749-97c3-acf61ef2dfd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:34.296515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f1f4f-4a3f-4749-97c3-acf61ef2dfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:34.296784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f1f4f-4a3f-4749-97c3-acf61ef2dfd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:34.297469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f1f4f-4a3f-4749-97c3-acf61ef2dfd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:34.299595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f1f4f-4a3f-4749-97c3-acf61ef2dfd1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:34.324705 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1101/2220] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:40:34 2022] POST /identity/v3/users => generated 296 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:40:34.330509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40d03b-ea92-4b15-a7da-175f7d5afa97 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:34.341403 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40d03b-ea92-4b15-a7da-175f7d5afa97 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:34.347345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884b205-ddd1-46dd-91b6-04adb0aea338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:34.348138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884b205-ddd1-46dd-91b6-04adb0aea338 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:34.348414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884b205-ddd1-46dd-91b6-04adb0aea338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:34.348684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884b205-ddd1-46dd-91b6-04adb0aea338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:34.349386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884b205-ddd1-46dd-91b6-04adb0aea338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:34.350976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884b205-ddd1-46dd-91b6-04adb0aea338 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:34.361661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1120/2221] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:34.365800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f1f4f-4a3f-4749-97c3-acf61ef2dfd1 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:34.376660 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f1f4f-4a3f-4749-97c3-acf61ef2dfd1 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:34.382252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681ec33-d75b-426c-a5ea-31fc46716033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:34.383230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681ec33-d75b-426c-a5ea-31fc46716033 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:34.383230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681ec33-d75b-426c-a5ea-31fc46716033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:34.383387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681ec33-d75b-426c-a5ea-31fc46716033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:34.384010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681ec33-d75b-426c-a5ea-31fc46716033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:34.385675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681ec33-d75b-426c-a5ea-31fc46716033 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:34.399151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1102/2222] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:40:34.403492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884b205-ddd1-46dd-91b6-04adb0aea338 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:34.416190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884b205-ddd1-46dd-91b6-04adb0aea338 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:34.421743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3161f-00db-4e6b-a22a-aa1eeaf447d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:34.422491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3161f-00db-4e6b-a22a-aa1eeaf447d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:34.422773 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3161f-00db-4e6b-a22a-aa1eeaf447d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:34.423223 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3161f-00db-4e6b-a22a-aa1eeaf447d5 admin admin] PATH_INFO: `/v3/projects/93d3c7aa0c314df0b59f2cb5e8ef43b6/users/e53128f13c044465b10caf2b1a2c5729/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:34.439595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3161f-00db-4e6b-a22a-aa1eeaf447d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d3c7aa0c314df0b59f2cb5e8ef43b6, user_id=e53128f13c044465b10caf2b1a2c5729, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:34.442973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3161f-00db-4e6b-a22a-aa1eeaf447d5 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:34.458733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1121/2223] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:40:34 2022] PUT /identity/v3/projects/93d3c7aa0c314df0b59f2cb5e8ef43b6/users/e53128f13c044465b10caf2b1a2c5729/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:40:34.471262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3ac8-65ba-4b1c-999b-bb8c255ea8e9 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:34.471647 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3ac8-65ba-4b1c-999b-bb8c255ea8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:34.472040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3ac8-65ba-4b1c-999b-bb8c255ea8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:34.500827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-dfaf3ac8-65ba-4b1c-999b-bb8c255ea8e9 None None] MFA Rules not processed for user `e53128f13c044465b10caf2b1a2c5729`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:34.502115 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-dfaf3ac8-65ba-4b1c-999b-bb8c255ea8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:34.599781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1103/2224] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:40:34.607545 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3161f-00db-4e6b-a22a-aa1eeaf447d5 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:40:34.609352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ccc3161f-00db-4e6b-a22a-aa1eeaf447d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:34.672176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3161f-00db-4e6b-a22a-aa1eeaf447d5 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:40:34.676183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2826b2-0a03-4ebf-bd66-9e1853573b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:40:34.676947 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2826b2-0a03-4ebf-bd66-9e1853573b5a service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:34.678035 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2826b2-0a03-4ebf-bd66-9e1853573b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:34.678035 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2826b2-0a03-4ebf-bd66-9e1853573b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:34.690171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2826b2-0a03-4ebf-bd66-9e1853573b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:40:34.691654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2826b2-0a03-4ebf-bd66-9e1853573b5a service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:40:34.699496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf56c5-1058-4326-8d9a-9ca8a52eb573 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:40:34.699803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf56c5-1058-4326-8d9a-9ca8a52eb573 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:40:34.700164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf56c5-1058-4326-8d9a-9ca8a52eb573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:40:34.701774 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1122/2225] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:40:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:40:34.721225 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-d0cf56c5-1058-4326-8d9a-9ca8a52eb573 None None] MFA Rules not processed for user `4f250207e16c4515bd9beb9ca4b1fde6`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:40:34.722448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d0cf56c5-1058-4326-8d9a-9ca8a52eb573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:40:34.746274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:34.801332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1104/2226] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:40:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:40:44.696871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:40:44.794035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=15 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:41:10.977524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5add8-6a28-410c-a132-0c07b4fc8d38 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:41:10.978048 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5add8-6a28-410c-a132-0c07b4fc8d38 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:41:10.978149 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5add8-6a28-410c-a132-0c07b4fc8d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:41:10.997649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5dd5add8-6a28-410c-a132-0c07b4fc8d38 None None] MFA Rules not processed for user `7d712e201f2c47a8a48b6e24559e0d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:41:10.998862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5dd5add8-6a28-410c-a132-0c07b4fc8d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:41:11.053740 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1123/2227] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:41:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:41:11.275589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf56c5-1058-4326-8d9a-9ca8a52eb573 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:41:11.339179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf56c5-1058-4326-8d9a-9ca8a52eb573 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:41:11.343724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:41:11.344351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:41:11.344629 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:41:11.344901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] PATH_INFO: `/v3/users/e53128f13c044465b10caf2b1a2c5729` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:41:11.346690 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e53128f13c044465b10caf2b1a2c5729)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:41:11.348611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:41:11.404881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e53128f13c044465b10caf2b1a2c5729'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:11.413202 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e53128f13c044465b10caf2b1a2c5729'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:11.413925 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e53128f13c044465b10caf2b1a2c5729'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:11.414688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e53128f13c044465b10caf2b1a2c5729'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:11.424671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e53128f13c044465b10caf2b1a2c5729'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:11.452666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1105/2228] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:41:11 2022] DELETE /identity/v3/users/e53128f13c044465b10caf2b1a2c5729 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:41:11.457558 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb103e17-f892-4f5b-8c87-41020e985f83 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:41:11.457807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb103e17-f892-4f5b-8c87-41020e985f83 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:41:11.458087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb103e17-f892-4f5b-8c87-41020e985f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:41:11.483355 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-cb103e17-f892-4f5b-8c87-41020e985f83 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:41:11.484661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-cb103e17-f892-4f5b-8c87-41020e985f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:41:11.557788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1124/2229] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:41:11.567716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:41:11.569313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:41:11.629691 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74b32a-01de-478c-8dae-bbbc2a159ed6 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:41:11.633298 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccc0c2-4045-4df6-b12c-e8a6880668f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:41:11.633902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccc0c2-4045-4df6-b12c-e8a6880668f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:41:11.634178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccc0c2-4045-4df6-b12c-e8a6880668f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:41:11.634449 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccc0c2-4045-4df6-b12c-e8a6880668f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:41:11.644310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dccc0c2-4045-4df6-b12c-e8a6880668f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:41:11.645990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dccc0c2-4045-4df6-b12c-e8a6880668f3 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:41:11.658456 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1106/2230] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:41:11.966337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb103e17-f892-4f5b-8c87-41020e985f83 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:41:11.967788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-cb103e17-f892-4f5b-8c87-41020e985f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:41:11.979651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb103e17-f892-4f5b-8c87-41020e985f83 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:41:11.984688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:41:11.985719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:41:11.985867 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:41:11.986320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] PATH_INFO: `/v3/projects/93d3c7aa0c314df0b59f2cb5e8ef43b6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:41:11.987721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=93d3c7aa0c314df0b59f2cb5e8ef43b6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:41:11.989850 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:41:12.045169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93d3c7aa0c314df0b59f2cb5e8ef43b6', 'request_id': 'req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:12.052170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93d3c7aa0c314df0b59f2cb5e8ef43b6', 'request_id': 'req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:12.058986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93d3c7aa0c314df0b59f2cb5e8ef43b6', 'request_id': 'req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:12.072470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] The token cache is being invalidate because project 93d3c7aa0c314df0b59f2cb5e8ef43b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:41:12.072743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:12.073435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:12.074983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1125/2231] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:41:11 2022] DELETE /identity/v3/projects/93d3c7aa0c314df0b59f2cb5e8ef43b6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:41:12.244514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccc0c2-4045-4df6-b12c-e8a6880668f3 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:41:12.246168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6dccc0c2-4045-4df6-b12c-e8a6880668f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:41:12.261215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccc0c2-4045-4df6-b12c-e8a6880668f3 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:41:12.281559 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:41:12.282229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:41:12.283465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:41:12.319406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:41:12.333885 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:41:12.334525 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:41:12.334998 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:41:12.335116 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] PATH_INFO: `/v3/users/2c43b1f9964f476bb63c51568779f08e` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:41:12.336218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c43b1f9964f476bb63c51568779f08e)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:41:12.338186 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:41:12.361248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ec8d7-189b-4897-86c0-fd26a2ab1c9e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:41:12.367502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bdb7b-899b-42d2-8099-26940d54af80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:41:12.368153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bdb7b-899b-42d2-8099-26940d54af80 service placement] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:41:12.368461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bdb7b-899b-42d2-8099-26940d54af80 service placement] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:41:12.368794 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bdb7b-899b-42d2-8099-26940d54af80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:41:12.371072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9c1bdb7b-899b-42d2-8099-26940d54af80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:41:12.383020 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bdb7b-899b-42d2-8099-26940d54af80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:41:12.384631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bdb7b-899b-42d2-8099-26940d54af80 service placement] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:41:12.385987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c43b1f9964f476bb63c51568779f08e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:12.394135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c43b1f9964f476bb63c51568779f08e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:12.394822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c43b1f9964f476bb63c51568779f08e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:12.395549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c43b1f9964f476bb63c51568779f08e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:12.395927 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1126/2232] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:41:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:41:12.403568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c43b1f9964f476bb63c51568779f08e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:12.423063 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1107/2233] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:41:12 2022] DELETE /identity/v3/users/2c43b1f9964f476bb63c51568779f08e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:41:12.427686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea94789-2436-406f-b034-807e7c580e1f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:41:12.427971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea94789-2436-406f-b034-807e7c580e1f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:41:12.428251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea94789-2436-406f-b034-807e7c580e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:41:12.447324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-0ea94789-2436-406f-b034-807e7c580e1f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:41:12.448989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-0ea94789-2436-406f-b034-807e7c580e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:41:12.526844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1127/2234] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:41:12.534240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:41:12.535930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:41:12.607340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde169c9-8a24-4397-b9e4-2d2e69c584b3 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:41:12.611433 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162f65c-0939-45e8-a0d5-5724a660c023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:41:12.612034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162f65c-0939-45e8-a0d5-5724a660c023 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:41:12.612362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162f65c-0939-45e8-a0d5-5724a660c023 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:41:12.612634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162f65c-0939-45e8-a0d5-5724a660c023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:41:12.622240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162f65c-0939-45e8-a0d5-5724a660c023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:41:12.623685 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162f65c-0939-45e8-a0d5-5724a660c023 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:41:12.633497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1108/2235] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:41:12.952036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea94789-2436-406f-b034-807e7c580e1f None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:41:12.953585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-0ea94789-2436-406f-b034-807e7c580e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:41:12.967838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea94789-2436-406f-b034-807e7c580e1f None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:41:12.972904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9408be5-760c-4b00-b0c5-5a1007771819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:41:12.973595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9408be5-760c-4b00-b0c5-5a1007771819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:41:12.973880 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9408be5-760c-4b00-b0c5-5a1007771819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:41:12.974166 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9408be5-760c-4b00-b0c5-5a1007771819 admin admin] PATH_INFO: `/v3/projects/f07839378483494990a49ffdea7dd95e` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:41:12.975331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9408be5-760c-4b00-b0c5-5a1007771819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f07839378483494990a49ffdea7dd95e)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:41:12.977288 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9408be5-760c-4b00-b0c5-5a1007771819 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:41:13.031493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-e9408be5-760c-4b00-b0c5-5a1007771819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f07839378483494990a49ffdea7dd95e', 'request_id': 'req-e9408be5-760c-4b00-b0c5-5a1007771819'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:13.038069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-e9408be5-760c-4b00-b0c5-5a1007771819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f07839378483494990a49ffdea7dd95e', 'request_id': 'req-e9408be5-760c-4b00-b0c5-5a1007771819'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:13.045506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-e9408be5-760c-4b00-b0c5-5a1007771819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f07839378483494990a49ffdea7dd95e', 'request_id': 'req-e9408be5-760c-4b00-b0c5-5a1007771819'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:13.059952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-e9408be5-760c-4b00-b0c5-5a1007771819 admin admin] The token cache is being invalidate because project f07839378483494990a49ffdea7dd95e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:41:13.060401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-e9408be5-760c-4b00-b0c5-5a1007771819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:13.061302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-e9408be5-760c-4b00-b0c5-5a1007771819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:41:13.062975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1128/2236] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:41:12 2022] DELETE /identity/v3/projects/f07839378483494990a49ffdea7dd95e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:41:22.629013 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:41:23.056312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:41:38.345622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162f65c-0939-45e8-a0d5-5724a660c023 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:41:38.347182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6162f65c-0939-45e8-a0d5-5724a660c023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:41:38.359678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162f65c-0939-45e8-a0d5-5724a660c023 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:41:38.411055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40710fab-cce0-4aa0-b5f6-19d52f9b4f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:41:38.411820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40710fab-cce0-4aa0-b5f6-19d52f9b4f2c service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:41:38.412100 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40710fab-cce0-4aa0-b5f6-19d52f9b4f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:41:38.412396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40710fab-cce0-4aa0-b5f6-19d52f9b4f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:41:38.414261 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-40710fab-cce0-4aa0-b5f6-19d52f9b4f2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:41:38.480258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40710fab-cce0-4aa0-b5f6-19d52f9b4f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:41:38.481757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40710fab-cce0-4aa0-b5f6-19d52f9b4f2c service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:41:38.491824 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1109/2237] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:41:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:41:48.487290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=50 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:42:22.438350 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850580b8-3747-4a75-8312-ffa8bed780a9 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:42:22.438878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850580b8-3747-4a75-8312-ffa8bed780a9 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:42:22.438878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850580b8-3747-4a75-8312-ffa8bed780a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:42:22.458226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-850580b8-3747-4a75-8312-ffa8bed780a9 None None] MFA Rules not processed for user `7e3b9ceb95374f1dbf356122c8ed463c`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:42:22.459540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-850580b8-3747-4a75-8312-ffa8bed780a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:42:22.515124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1129/2238] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:42:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:42:32.507875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=25 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:42:57.109634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40710fab-cce0-4aa0-b5f6-19d52f9b4f2c service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:42:57.111036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-40710fab-cce0-4aa0-b5f6-19d52f9b4f2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:42:57.122828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40710fab-cce0-4aa0-b5f6-19d52f9b4f2c service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:42:57.126496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c32a60-a25e-43f3-8ef4-c89dd36ccad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:42:57.127087 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c32a60-a25e-43f3-8ef4-c89dd36ccad5 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:42:57.127366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c32a60-a25e-43f3-8ef4-c89dd36ccad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:42:57.127637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c32a60-a25e-43f3-8ef4-c89dd36ccad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:42:57.129338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d1c32a60-a25e-43f3-8ef4-c89dd36ccad5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:42:57.208556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c32a60-a25e-43f3-8ef4-c89dd36ccad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:42:57.210204 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c32a60-a25e-43f3-8ef4-c89dd36ccad5 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:42:57.221330 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1110/2239] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:42:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:42:58.404110 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850580b8-3747-4a75-8312-ffa8bed780a9 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:42:58.413931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850580b8-3747-4a75-8312-ffa8bed780a9 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:42:58.417501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f107841-77cd-42a3-917d-d31305752078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:42:58.418390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f107841-77cd-42a3-917d-d31305752078 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:42:58.418670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f107841-77cd-42a3-917d-d31305752078 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:42:58.418944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f107841-77cd-42a3-917d-d31305752078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:42:58.420731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9f107841-77cd-42a3-917d-d31305752078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:42:58.434238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f107841-77cd-42a3-917d-d31305752078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:42:58.435832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f107841-77cd-42a3-917d-d31305752078 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:42:58.447714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1130/2240] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:42:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:43:07.216278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=34 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:43:07.538495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019a10f-9357-488c-8c8e-f05407a4d9ba None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:07.538785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019a10f-9357-488c-8c8e-f05407a4d9ba None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:07.539065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019a10f-9357-488c-8c8e-f05407a4d9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:07.560381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-0019a10f-9357-488c-8c8e-f05407a4d9ba None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:43:07.561630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-0019a10f-9357-488c-8c8e-f05407a4d9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:07.616945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1111/2241] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:43:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:43:07.663010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f107841-77cd-42a3-917d-d31305752078 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:43:07.664416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9f107841-77cd-42a3-917d-d31305752078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:07.733555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f107841-77cd-42a3-917d-d31305752078 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:43:07.738614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8aa5b6-56dc-4e9e-b119-c09ebd182460 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:43:07.739231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8aa5b6-56dc-4e9e-b119-c09ebd182460 service swift] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:07.739512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8aa5b6-56dc-4e9e-b119-c09ebd182460 service swift] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:07.739788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8aa5b6-56dc-4e9e-b119-c09ebd182460 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:07.748701 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8aa5b6-56dc-4e9e-b119-c09ebd182460 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:43:07.750399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8aa5b6-56dc-4e9e-b119-c09ebd182460 service swift] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:43:07.758417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1131/2242] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:43:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:43:15.621881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019a10f-9357-488c-8c8e-f05407a4d9ba None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:43:15.623737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-0019a10f-9357-488c-8c8e-f05407a4d9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:15.691226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019a10f-9357-488c-8c8e-f05407a4d9ba None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:43:15.695000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962ec06-5982-44e4-8e6f-67981f982f92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:43:15.695753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962ec06-5982-44e4-8e6f-67981f982f92 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:15.696036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962ec06-5982-44e4-8e6f-67981f982f92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:15.696307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962ec06-5982-44e4-8e6f-67981f982f92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:15.699373 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e962ec06-5982-44e4-8e6f-67981f982f92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:15.776196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962ec06-5982-44e4-8e6f-67981f982f92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:43:15.778244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962ec06-5982-44e4-8e6f-67981f982f92 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:43:15.788491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1112/2243] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:43:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:43:17.755154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=38 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:43:22.998184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8aa5b6-56dc-4e9e-b119-c09ebd182460 service swift] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:43:23.007741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8aa5b6-56dc-4e9e-b119-c09ebd182460 service swift] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:43:23.011232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e9750-9c29-4d0c-9cf8-01df6e3ec719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:43:23.011820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e9750-9c29-4d0c-9cf8-01df6e3ec719 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:23.012242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e9750-9c29-4d0c-9cf8-01df6e3ec719 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:23.012529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e9750-9c29-4d0c-9cf8-01df6e3ec719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:23.014241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b77e9750-9c29-4d0c-9cf8-01df6e3ec719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:23.023438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e9750-9c29-4d0c-9cf8-01df6e3ec719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:43:23.024821 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e9750-9c29-4d0c-9cf8-01df6e3ec719 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:43:23.032592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1132/2244] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:43:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:43:23.155977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261fad5-1a21-41c4-bb0d-3747b0db6c32 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:23.156548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261fad5-1a21-41c4-bb0d-3747b0db6c32 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:23.156676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261fad5-1a21-41c4-bb0d-3747b0db6c32 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:23.158159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1113/2245] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:43:23.168822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbc149-45aa-415e-9ac4-05925540f161 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:23.169250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbc149-45aa-415e-9ac4-05925540f161 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:23.169603 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbc149-45aa-415e-9ac4-05925540f161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:23.191472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-1cfbc149-45aa-415e-9ac4-05925540f161 None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:43:23.192695 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1cfbc149-45aa-415e-9ac4-05925540f161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:23.202781 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1133/2246] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:43:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:43:23.217464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261fad5-1a21-41c4-bb0d-3747b0db6c32 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:43:23.219191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c261fad5-1a21-41c4-bb0d-3747b0db6c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:23.282001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261fad5-1a21-41c4-bb0d-3747b0db6c32 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:43:23.285550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac6699-09a0-4956-a433-9c3b2f9e0bed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff22c2632c614fca8008feab93c8fa2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:43:23.286151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac6699-09a0-4956-a433-9c3b2f9e0bed service glance] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:23.286425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac6699-09a0-4956-a433-9c3b2f9e0bed service glance] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:23.286696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac6699-09a0-4956-a433-9c3b2f9e0bed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:23.298084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac6699-09a0-4956-a433-9c3b2f9e0bed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:43:23.299502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac6699-09a0-4956-a433-9c3b2f9e0bed service glance] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:43:23.311126 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1114/2247] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:43:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:43:23.384279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4866a-64ae-4b30-bae8-d5e08d27f1ab None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:23.384484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4866a-64ae-4b30-bae8-d5e08d27f1ab None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:23.384780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4866a-64ae-4b30-bae8-d5e08d27f1ab None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:23.386347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1134/2248] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:43:23.392261 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22c901-3ee0-4bd2-a404-b7ee2797235d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:23.392488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22c901-3ee0-4bd2-a404-b7ee2797235d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:23.392762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22c901-3ee0-4bd2-a404-b7ee2797235d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:23.401188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:43:23.413773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-1e22c901-3ee0-4bd2-a404-b7ee2797235d None None] MFA Rules not processed for user `736efafe9fc94b45bc75566b1696c498`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:43:23.414977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1e22c901-3ee0-4bd2-a404-b7ee2797235d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:23.422511 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1115/2249] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:43:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:43:23.434850 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4866a-64ae-4b30-bae8-d5e08d27f1ab None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:43:23.436547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-21b4866a-64ae-4b30-bae8-d5e08d27f1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:23.449710 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4866a-64ae-4b30-bae8-d5e08d27f1ab None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:43:23.453317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b91b6-d187-458c-8891-60496171cb01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:43:23.453942 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b91b6-d187-458c-8891-60496171cb01 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:23.454219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b91b6-d187-458c-8891-60496171cb01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:23.454516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b91b6-d187-458c-8891-60496171cb01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:23.461653 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b91b6-d187-458c-8891-60496171cb01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:43:23.463555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b91b6-d187-458c-8891-60496171cb01 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:43:23.471214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1135/2250] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:43:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:43:23.554775 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d7649-f8af-496e-8c3d-033c5114cde1 None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:23.555061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d7649-f8af-496e-8c3d-033c5114cde1 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:23.555333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d7649-f8af-496e-8c3d-033c5114cde1 None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:23.556725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1116/2251] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:43:23.561615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023b31a-4bb6-4688-aa4e-573e43f54964 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:23.561903 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023b31a-4bb6-4688-aa4e-573e43f54964 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:23.562176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023b31a-4bb6-4688-aa4e-573e43f54964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:23.581736 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-7023b31a-4bb6-4688-aa4e-573e43f54964 None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:43:23.582958 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7023b31a-4bb6-4688-aa4e-573e43f54964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:23.592706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1136/2252] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:43:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:43:23.603322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d7649-f8af-496e-8c3d-033c5114cde1 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:43:23.615049 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d7649-f8af-496e-8c3d-033c5114cde1 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:43:23.618594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd563cd3-4630-4a3b-b8f7-7e0abd8ce833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:43:23.619179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd563cd3-4630-4a3b-b8f7-7e0abd8ce833 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:23.619453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd563cd3-4630-4a3b-b8f7-7e0abd8ce833 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:23.619736 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd563cd3-4630-4a3b-b8f7-7e0abd8ce833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:23.630206 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd563cd3-4630-4a3b-b8f7-7e0abd8ce833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:43:23.631606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd563cd3-4630-4a3b-b8f7-7e0abd8ce833 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:43:23.642845 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1117/2253] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:43:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:43:23.713793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023b31a-4bb6-4688-aa4e-573e43f54964 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:43:23.715329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7023b31a-4bb6-4688-aa4e-573e43f54964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:23.785038 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023b31a-4bb6-4688-aa4e-573e43f54964 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:43:23.790196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dda5a7-aeec-4b5c-979c-830c42c00d4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:43:23.790804 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dda5a7-aeec-4b5c-979c-830c42c00d4d service placement] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:23.791111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dda5a7-aeec-4b5c-979c-830c42c00d4d service placement] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:23.791386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dda5a7-aeec-4b5c-979c-830c42c00d4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:23.793164 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-73dda5a7-aeec-4b5c-979c-830c42c00d4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:23.806707 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dda5a7-aeec-4b5c-979c-830c42c00d4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:43:23.808326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dda5a7-aeec-4b5c-979c-830c42c00d4d service placement] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:43:23.818454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1137/2254] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:43:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:43:26.796575 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e4b39-f8e9-4751-abc2-98103f33dad1 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:26.797140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e4b39-f8e9-4751-abc2-98103f33dad1 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:26.797295 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e4b39-f8e9-4751-abc2-98103f33dad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:26.816335 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ce6e4b39-f8e9-4751-abc2-98103f33dad1 None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:43:26.817581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ce6e4b39-f8e9-4751-abc2-98103f33dad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:26.825209 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1118/2255] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:43:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:43:26.837181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dda5a7-aeec-4b5c-979c-830c42c00d4d service placement] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:43:26.848796 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dda5a7-aeec-4b5c-979c-830c42c00d4d service placement] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:43:26.854078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816c965-d5cb-42aa-9ec4-9a918e3f5e38 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:43:26.854681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816c965-d5cb-42aa-9ec4-9a918e3f5e38 service swift] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:26.854961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816c965-d5cb-42aa-9ec4-9a918e3f5e38 service swift] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:26.855232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816c965-d5cb-42aa-9ec4-9a918e3f5e38 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:26.863784 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816c965-d5cb-42aa-9ec4-9a918e3f5e38 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:43:26.865540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816c965-d5cb-42aa-9ec4-9a918e3f5e38 service swift] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:43:26.874336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1138/2256] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:43:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:43:36.811428 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=11 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:43:36.870676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=56 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:43:44.986538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e4b39-f8e9-4751-abc2-98103f33dad1 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:43:45.000496 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e4b39-f8e9-4751-abc2-98103f33dad1 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:43:45.004456 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ec282-fddf-483c-9292-5ae4aa3a494e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:43:45.005059 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ec282-fddf-483c-9292-5ae4aa3a494e service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:45.005378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ec282-fddf-483c-9292-5ae4aa3a494e service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:45.005642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ec282-fddf-483c-9292-5ae4aa3a494e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:45.008343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3a6ec282-fddf-483c-9292-5ae4aa3a494e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:45.019442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ec282-fddf-483c-9292-5ae4aa3a494e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:43:45.020907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ec282-fddf-483c-9292-5ae4aa3a494e service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:43:45.031839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1119/2257] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:43:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:43:48.749308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05a22a-7db2-4ff9-a4d9-3810bc43591b None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:48.749804 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05a22a-7db2-4ff9-a4d9-3810bc43591b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:48.749925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05a22a-7db2-4ff9-a4d9-3810bc43591b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:48.769879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-da05a22a-7db2-4ff9-a4d9-3810bc43591b None None] MFA Rules not processed for user `7d712e201f2c47a8a48b6e24559e0d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:43:48.771208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-da05a22a-7db2-4ff9-a4d9-3810bc43591b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:48.778675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1139/2258] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:43:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:43:55.025686 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=6 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:43:56.520520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ec282-fddf-483c-9292-5ae4aa3a494e service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:43:56.522804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3a6ec282-fddf-483c-9292-5ae4aa3a494e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:56.543588 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ec282-fddf-483c-9292-5ae4aa3a494e service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:43:56.551769 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2d297-d7fc-4db1-9b94-6f1000f1cd3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:43:56.552667 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2d297-d7fc-4db1-9b94-6f1000f1cd3a service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:56.553175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2d297-d7fc-4db1-9b94-6f1000f1cd3a service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:56.553730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2d297-d7fc-4db1-9b94-6f1000f1cd3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:56.555849 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9ea2d297-d7fc-4db1-9b94-6f1000f1cd3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:56.576427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2d297-d7fc-4db1-9b94-6f1000f1cd3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:43:56.578335 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2d297-d7fc-4db1-9b94-6f1000f1cd3a service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:43:56.604462 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1120/2259] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:43:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:43:57.173673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e4b58-fb9f-4bd1-a324-8c9aba4807d5 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:57.176438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e4b58-fb9f-4bd1-a324-8c9aba4807d5 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:57.177475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e4b58-fb9f-4bd1-a324-8c9aba4807d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:57.205714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-7d4e4b58-fb9f-4bd1-a324-8c9aba4807d5 None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:43:57.207150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7d4e4b58-fb9f-4bd1-a324-8c9aba4807d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:43:57.224533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1140/2260] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 08:43:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:43:57.238716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2d297-d7fc-4db1-9b94-6f1000f1cd3a service nova] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:43:57.253487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2d297-d7fc-4db1-9b94-6f1000f1cd3a service nova] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:43:57.267787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369871dd-d7c6-430f-a36e-445d79c58ee4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:43:57.268444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369871dd-d7c6-430f-a36e-445d79c58ee4 service swift] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:43:57.268714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369871dd-d7c6-430f-a36e-445d79c58ee4 service swift] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:43:57.269358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369871dd-d7c6-430f-a36e-445d79c58ee4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:43:57.278817 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369871dd-d7c6-430f-a36e-445d79c58ee4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:43:57.280426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369871dd-d7c6-430f-a36e-445d79c58ee4 service swift] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:43:57.290219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1121/2261] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:43:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 08:44:07.197689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=6 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:44:07.286016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=12 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:44:15.468182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e4b58-fb9f-4bd1-a324-8c9aba4807d5 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:44:15.481802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e4b58-fb9f-4bd1-a324-8c9aba4807d5 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:44:15.487252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d7d96-a495-428f-86e0-29d2a58032e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:44:15.487844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d7d96-a495-428f-86e0-29d2a58032e9 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:44:15.488117 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d7d96-a495-428f-86e0-29d2a58032e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:44:15.488385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d7d96-a495-428f-86e0-29d2a58032e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:44:15.490144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-456d7d96-a495-428f-86e0-29d2a58032e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:44:15.549985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d7d96-a495-428f-86e0-29d2a58032e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:44:15.551569 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d7d96-a495-428f-86e0-29d2a58032e9 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:44:15.561413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1141/2262] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:44:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:44:19.493280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369871dd-d7c6-430f-a36e-445d79c58ee4 service swift] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:44:19.494689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-369871dd-d7c6-430f-a36e-445d79c58ee4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:44:19.511268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369871dd-d7c6-430f-a36e-445d79c58ee4 service swift] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:44:19.521532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d873a-14b2-4070-aa30-0485f56f4592 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:44:19.522241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d873a-14b2-4070-aa30-0485f56f4592 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:44:19.522596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d873a-14b2-4070-aa30-0485f56f4592 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:44:19.522942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d873a-14b2-4070-aa30-0485f56f4592 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:44:19.524772 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c94d873a-14b2-4070-aa30-0485f56f4592 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:44:19.536681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d873a-14b2-4070-aa30-0485f56f4592 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:44:19.538308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d873a-14b2-4070-aa30-0485f56f4592 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:44:19.548724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1122/2263] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:44:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:44:25.556428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=28 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:44:29.544760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=6 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:44:32.845451 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d7d96-a495-428f-86e0-29d2a58032e9 service nova] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:44:32.855273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d7d96-a495-428f-86e0-29d2a58032e9 service nova] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:44:32.858920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346d3ac-0de1-448b-a38c-fec177dd4361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:44:32.859517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346d3ac-0de1-448b-a38c-fec177dd4361 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:44:32.859814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346d3ac-0de1-448b-a38c-fec177dd4361 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:44:32.860076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346d3ac-0de1-448b-a38c-fec177dd4361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:44:32.861828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e346d3ac-0de1-448b-a38c-fec177dd4361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:44:32.872259 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346d3ac-0de1-448b-a38c-fec177dd4361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:44:32.873708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346d3ac-0de1-448b-a38c-fec177dd4361 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:44:32.881883 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1142/2264] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:44:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:44:34.051563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acd3b6-70c6-4168-8de5-0db5af45e3db None None] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:44:34.052102 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acd3b6-70c6-4168-8de5-0db5af45e3db None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:44:34.052362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acd3b6-70c6-4168-8de5-0db5af45e3db None None] PATH_INFO: `/` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:44:34.054924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1123/2265] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:44:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:44:34.061378 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9e660-c4d7-4ddb-a322-d50d8947bc10 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:44:34.061901 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9e660-c4d7-4ddb-a322-d50d8947bc10 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:44:34.062222 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9e660-c4d7-4ddb-a322-d50d8947bc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:44:34.084884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-a2c9e660-c4d7-4ddb-a322-d50d8947bc10 None None] MFA Rules not processed for user `736efafe9fc94b45bc75566b1696c498`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:44:34.086140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a2c9e660-c4d7-4ddb-a322-d50d8947bc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:44:34.093116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1143/2266] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:44:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:44:34.107963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acd3b6-70c6-4168-8de5-0db5af45e3db None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:44:34.120953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acd3b6-70c6-4168-8de5-0db5af45e3db None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:44:34.126609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c913d-3220-4172-a47a-81156cfa8741 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:44:34.127243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c913d-3220-4172-a47a-81156cfa8741 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:44:34.127579 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c913d-3220-4172-a47a-81156cfa8741 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:44:34.127785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c913d-3220-4172-a47a-81156cfa8741 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:44:34.136778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c913d-3220-4172-a47a-81156cfa8741 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:44:34.138338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c913d-3220-4172-a47a-81156cfa8741 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:44:34.146390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1124/2267] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:44:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:44:34.245521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f3e33-10ef-4e84-a42a-0c37e47ac118 None None] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:44:34.246025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f3e33-10ef-4e84-a42a-0c37e47ac118 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:44:34.246304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f3e33-10ef-4e84-a42a-0c37e47ac118 None None] PATH_INFO: `/` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:44:34.247756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1144/2268] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 08:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 08:44:34.253911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd0ad-8ece-44d2-855a-71ea53c1e35c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:44:34.254208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd0ad-8ece-44d2-855a-71ea53c1e35c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:44:34.254551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd0ad-8ece-44d2-855a-71ea53c1e35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:44:34.277925 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-f90cd0ad-8ece-44d2-855a-71ea53c1e35c None None] MFA Rules not processed for user `9c3c6efc3c154ff89066035cd9583459`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:44:34.279148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f90cd0ad-8ece-44d2-855a-71ea53c1e35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:44:34.288961 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1125/2269] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 08:44:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:44:34.300723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f3e33-10ef-4e84-a42a-0c37e47ac118 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:44:34.313823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f3e33-10ef-4e84-a42a-0c37e47ac118 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:44:34.317553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193bc84-846f-4eab-b6dc-71d74b1acc29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:44:34.318172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193bc84-846f-4eab-b6dc-71d74b1acc29 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:44:34.318451 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193bc84-846f-4eab-b6dc-71d74b1acc29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:44:34.318822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193bc84-846f-4eab-b6dc-71d74b1acc29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:44:34.329936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193bc84-846f-4eab-b6dc-71d74b1acc29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:44:34.331509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193bc84-846f-4eab-b6dc-71d74b1acc29 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:44:34.344744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1145/2270] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:44:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:44:35.404475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c850-5b5e-45e4-b687-e35cae408570 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:44:35.405133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c850-5b5e-45e4-b687-e35cae408570 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:44:35.405133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344c850-5b5e-45e4-b687-e35cae408570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:44:35.428340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-1344c850-5b5e-45e4-b687-e35cae408570 None None] MFA Rules not processed for user `7e3b9ceb95374f1dbf356122c8ed463c`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:44:35.429705 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1344c850-5b5e-45e4-b687-e35cae408570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:44:35.443105 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1126/2271] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:44:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:44:35.641280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193bc84-846f-4eab-b6dc-71d74b1acc29 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:44:35.652834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193bc84-846f-4eab-b6dc-71d74b1acc29 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:44:35.661992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e51d4-75b0-4743-b83e-ddd55bec7065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:44:35.662612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e51d4-75b0-4743-b83e-ddd55bec7065 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:44:35.662893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e51d4-75b0-4743-b83e-ddd55bec7065 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:44:35.663169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e51d4-75b0-4743-b83e-ddd55bec7065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:44:35.665005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-085e51d4-75b0-4743-b83e-ddd55bec7065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:44:35.679556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e51d4-75b0-4743-b83e-ddd55bec7065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:44:35.681170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e51d4-75b0-4743-b83e-ddd55bec7065 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:44:35.693733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1146/2272] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:44:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:44:45.422154 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=12 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:44:45.688349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=21 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:44:49.507351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c850-5b5e-45e4-b687-e35cae408570 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:44:49.528311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344c850-5b5e-45e4-b687-e35cae408570 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:44:49.533975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe6fce-f4a0-4268-a02f-15eab6d807e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:44:49.534579 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe6fce-f4a0-4268-a02f-15eab6d807e8 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:44:49.534848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe6fce-f4a0-4268-a02f-15eab6d807e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:44:49.535117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe6fce-f4a0-4268-a02f-15eab6d807e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:44:49.537074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e2fe6fce-f4a0-4268-a02f-15eab6d807e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:44:49.550216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe6fce-f4a0-4268-a02f-15eab6d807e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:44:49.551813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe6fce-f4a0-4268-a02f-15eab6d807e8 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:44:49.563146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1127/2273] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:44:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:44:59.558609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=6 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:18.314745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa864d26-53b9-4fd1-93d0-7b04a2eb7b8d None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:18.315236 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa864d26-53b9-4fd1-93d0-7b04a2eb7b8d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:18.315332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa864d26-53b9-4fd1-93d0-7b04a2eb7b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:18.337854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-aa864d26-53b9-4fd1-93d0-7b04a2eb7b8d None None] MFA Rules not processed for user `b0cbf49cfca74838bee4c8162dfe27ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:18.339126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-aa864d26-53b9-4fd1-93d0-7b04a2eb7b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:18.395779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1147/2274] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:18.407221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe6fce-f4a0-4268-a02f-15eab6d807e8 service nova] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:18.418840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe6fce-f4a0-4268-a02f-15eab6d807e8 service nova] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:18.424654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdba311-ebe7-4cb0-b4ee-81f6819272cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:18.425285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdba311-ebe7-4cb0-b4ee-81f6819272cc service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:18.425567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdba311-ebe7-4cb0-b4ee-81f6819272cc service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:18.425839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdba311-ebe7-4cb0-b4ee-81f6819272cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:18.427517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6bdba311-ebe7-4cb0-b4ee-81f6819272cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:18.438355 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdba311-ebe7-4cb0-b4ee-81f6819272cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:18.439937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdba311-ebe7-4cb0-b4ee-81f6819272cc service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:18.448845 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1128/2275] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:28.388957 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=25 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:28.444684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=6 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:28.751852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa864d26-53b9-4fd1-93d0-7b04a2eb7b8d None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:28.761517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa864d26-53b9-4fd1-93d0-7b04a2eb7b8d None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:28.765625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd240d-d409-429c-a1cf-ed90e719240b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:28.766281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd240d-d409-429c-a1cf-ed90e719240b service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:28.766668 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd240d-d409-429c-a1cf-ed90e719240b service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:28.767056 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd240d-d409-429c-a1cf-ed90e719240b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:28.769012 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-51fd240d-d409-429c-a1cf-ed90e719240b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:28.835272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd240d-d409-429c-a1cf-ed90e719240b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:28.836030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd240d-d409-429c-a1cf-ed90e719240b service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:28.847286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1148/2276] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:32.466936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdba311-ebe7-4cb0-b4ee-81f6819272cc service nova] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:32.478326 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdba311-ebe7-4cb0-b4ee-81f6819272cc service nova] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:32.483892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916dd02-91d7-4961-8b0e-619b9ea4dcfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:32.484690 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916dd02-91d7-4961-8b0e-619b9ea4dcfd service placement] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:32.484963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916dd02-91d7-4961-8b0e-619b9ea4dcfd service placement] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:32.485269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916dd02-91d7-4961-8b0e-619b9ea4dcfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:32.487189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5916dd02-91d7-4961-8b0e-619b9ea4dcfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:32.499867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916dd02-91d7-4961-8b0e-619b9ea4dcfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:32.501532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916dd02-91d7-4961-8b0e-619b9ea4dcfd service placement] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:32.517740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1129/2277] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:32.519780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd240d-d409-429c-a1cf-ed90e719240b service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:32.521270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-51fd240d-d409-429c-a1cf-ed90e719240b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:32.535146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd240d-d409-429c-a1cf-ed90e719240b service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:32.540151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3a2a8-08d5-461b-9d6e-4863c5e3e3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:32.540820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3a2a8-08d5-461b-9d6e-4863c5e3e3cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:32.541214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3a2a8-08d5-461b-9d6e-4863c5e3e3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:32.542065 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3a2a8-08d5-461b-9d6e-4863c5e3e3cf admin admin] PATH_INFO: `/v3/users/7d712e201f2c47a8a48b6e24559e0d60` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:32.543367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3a2a8-08d5-461b-9d6e-4863c5e3e3cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d712e201f2c47a8a48b6e24559e0d60)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:32.545555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3a2a8-08d5-461b-9d6e-4863c5e3e3cf admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:32.597940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-35a3a2a8-08d5-461b-9d6e-4863c5e3e3cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d712e201f2c47a8a48b6e24559e0d60'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:32.610699 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-35a3a2a8-08d5-461b-9d6e-4863c5e3e3cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d712e201f2c47a8a48b6e24559e0d60'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:32.611395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-35a3a2a8-08d5-461b-9d6e-4863c5e3e3cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d712e201f2c47a8a48b6e24559e0d60'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:32.618394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-35a3a2a8-08d5-461b-9d6e-4863c5e3e3cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d712e201f2c47a8a48b6e24559e0d60'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:32.619019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-35a3a2a8-08d5-461b-9d6e-4863c5e3e3cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d712e201f2c47a8a48b6e24559e0d60'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:32.632780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1149/2278] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:45:32 2022] DELETE /identity/v3/users/7d712e201f2c47a8a48b6e24559e0d60 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:32.814972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916dd02-91d7-4961-8b0e-619b9ea4dcfd service placement] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:32.828182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916dd02-91d7-4961-8b0e-619b9ea4dcfd service placement] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:32.889076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d2c4d-1a2f-4d46-87fd-9097504a1985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:32.889745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d2c4d-1a2f-4d46-87fd-9097504a1985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:32.890030 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d2c4d-1a2f-4d46-87fd-9097504a1985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:32.890315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d2c4d-1a2f-4d46-87fd-9097504a1985 admin admin] PATH_INFO: `/v3/projects/42699952850d49fdafb670aeb5f613ed` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:32.891442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d2c4d-1a2f-4d46-87fd-9097504a1985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42699952850d49fdafb670aeb5f613ed)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:32.893380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d2c4d-1a2f-4d46-87fd-9097504a1985 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:32.944989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-302d2c4d-1a2f-4d46-87fd-9097504a1985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42699952850d49fdafb670aeb5f613ed', 'request_id': 'req-302d2c4d-1a2f-4d46-87fd-9097504a1985'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:32.951148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-302d2c4d-1a2f-4d46-87fd-9097504a1985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42699952850d49fdafb670aeb5f613ed', 'request_id': 'req-302d2c4d-1a2f-4d46-87fd-9097504a1985'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:32.960159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-302d2c4d-1a2f-4d46-87fd-9097504a1985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42699952850d49fdafb670aeb5f613ed', 'request_id': 'req-302d2c4d-1a2f-4d46-87fd-9097504a1985'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:32.974157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-302d2c4d-1a2f-4d46-87fd-9097504a1985 admin admin] The token cache is being invalidate because project 42699952850d49fdafb670aeb5f613ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:45:32.974445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-302d2c4d-1a2f-4d46-87fd-9097504a1985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:32.975102 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-302d2c4d-1a2f-4d46-87fd-9097504a1985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:32.976591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1130/2279] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:45:32 2022] DELETE /identity/v3/projects/42699952850d49fdafb670aeb5f613ed => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:32.981617 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28b2af-c437-48c7-be0e-6b5827fc2e14 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:32.981905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28b2af-c437-48c7-be0e-6b5827fc2e14 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:32.982182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28b2af-c437-48c7-be0e-6b5827fc2e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.002027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-8f28b2af-c437-48c7-be0e-6b5827fc2e14 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:33.003229 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8f28b2af-c437-48c7-be0e-6b5827fc2e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:33.062753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1150/2280] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:33.070468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190a27f-3352-4fe8-a16b-d815ee29970f None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.070760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190a27f-3352-4fe8-a16b-d815ee29970f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.071035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190a27f-3352-4fe8-a16b-d815ee29970f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.091719 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-4190a27f-3352-4fe8-a16b-d815ee29970f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:33.092917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4190a27f-3352-4fe8-a16b-d815ee29970f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:33.101006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1131/2281] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:33.105700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28b2af-c437-48c7-be0e-6b5827fc2e14 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:33.115566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28b2af-c437-48c7-be0e-6b5827fc2e14 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:33.120771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9353e2-d11f-43f7-922e-0b8e0cb116d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:33.121395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9353e2-d11f-43f7-922e-0b8e0cb116d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.121676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9353e2-d11f-43f7-922e-0b8e0cb116d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.121954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9353e2-d11f-43f7-922e-0b8e0cb116d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.122604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9353e2-d11f-43f7-922e-0b8e0cb116d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:45:33.122909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9353e2-d11f-43f7-922e-0b8e0cb116d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:33.124492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9353e2-d11f-43f7-922e-0b8e0cb116d9 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:33.125997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:33.139365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1151/2282] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:45:33.143576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190a27f-3352-4fe8-a16b-d815ee29970f None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:33.152733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190a27f-3352-4fe8-a16b-d815ee29970f None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:33.157388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcf801-d4ed-454b-9bc1-aced4a7d0bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:33.157992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcf801-d4ed-454b-9bc1-aced4a7d0bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.158266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcf801-d4ed-454b-9bc1-aced4a7d0bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.158827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcf801-d4ed-454b-9bc1-aced4a7d0bc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.159192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dcf801-d4ed-454b-9bc1-aced4a7d0bc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:33.161180 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dcf801-d4ed-454b-9bc1-aced4a7d0bc1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:33.199510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1132/2283] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:33 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:33.201199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9353e2-d11f-43f7-922e-0b8e0cb116d9 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:33.211398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9353e2-d11f-43f7-922e-0b8e0cb116d9 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:33.270374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cb947-f78e-45bc-85a7-494441395ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:33.270989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cb947-f78e-45bc-85a7-494441395ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.271264 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cb947-f78e-45bc-85a7-494441395ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.271537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cb947-f78e-45bc-85a7-494441395ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.272215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cb947-f78e-45bc-85a7-494441395ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:33.274154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cb947-f78e-45bc-85a7-494441395ec6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:33.300652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1152/2284] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:45:33 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:33.302207 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcf801-d4ed-454b-9bc1-aced4a7d0bc1 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:33.312776 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcf801-d4ed-454b-9bc1-aced4a7d0bc1 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:33.317700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e636d-fc5b-4c2a-b080-76f3966ec2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:33.318384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e636d-fc5b-4c2a-b080-76f3966ec2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.318793 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e636d-fc5b-4c2a-b080-76f3966ec2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.319198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e636d-fc5b-4c2a-b080-76f3966ec2e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.319950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e636d-fc5b-4c2a-b080-76f3966ec2e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:33.321843 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e636d-fc5b-4c2a-b080-76f3966ec2e3 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:33.332799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1133/2285] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:33.336812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cb947-f78e-45bc-85a7-494441395ec6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:33.347763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cb947-f78e-45bc-85a7-494441395ec6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:33.353017 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a8047-8c84-4160-af48-2a44469a3d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:33.353604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a8047-8c84-4160-af48-2a44469a3d91 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.354548 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a8047-8c84-4160-af48-2a44469a3d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.354828 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a8047-8c84-4160-af48-2a44469a3d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.355488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a8047-8c84-4160-af48-2a44469a3d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:33.357204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a8047-8c84-4160-af48-2a44469a3d91 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:33.369127 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1153/2286] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:33.372935 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e636d-fc5b-4c2a-b080-76f3966ec2e3 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:33.382875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e636d-fc5b-4c2a-b080-76f3966ec2e3 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:33.387736 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf7691-8b41-47c2-a1fb-9c09c7e7489d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:33.388381 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf7691-8b41-47c2-a1fb-9c09c7e7489d admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.388661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf7691-8b41-47c2-a1fb-9c09c7e7489d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.388934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf7691-8b41-47c2-a1fb-9c09c7e7489d admin admin] PATH_INFO: `/v3/projects/1c568697ed76457e83d37e7ad9aee06a/users/1d60f9a9799e443186ac551499c2805b/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.403460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf7691-8b41-47c2-a1fb-9c09c7e7489d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c568697ed76457e83d37e7ad9aee06a, user_id=1d60f9a9799e443186ac551499c2805b, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:33.406164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf7691-8b41-47c2-a1fb-9c09c7e7489d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:33.419420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1134/2287] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:33 2022] PUT /identity/v3/projects/1c568697ed76457e83d37e7ad9aee06a/users/1d60f9a9799e443186ac551499c2805b/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:33.430888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45553e71-5f3d-4f8f-949d-e7a293cdd5ba None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.431281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45553e71-5f3d-4f8f-949d-e7a293cdd5ba None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.431563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45553e71-5f3d-4f8f-949d-e7a293cdd5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.459347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-45553e71-5f3d-4f8f-949d-e7a293cdd5ba None None] MFA Rules not processed for user `1d60f9a9799e443186ac551499c2805b`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:33.460618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-45553e71-5f3d-4f8f-949d-e7a293cdd5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:33.543051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1154/2288] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:33 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:33.548175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf7691-8b41-47c2-a1fb-9c09c7e7489d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:33.573694 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:33.636424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf7691-8b41-47c2-a1fb-9c09c7e7489d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:33.641266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c938f-5a73-4642-b7eb-52e447573dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:33.641906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c938f-5a73-4642-b7eb-52e447573dcd admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.642188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c938f-5a73-4642-b7eb-52e447573dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.642466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c938f-5a73-4642-b7eb-52e447573dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.643112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c938f-5a73-4642-b7eb-52e447573dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:33.645016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c938f-5a73-4642-b7eb-52e447573dcd admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:33.672733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1135/2289] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:33 2022] POST /identity/v3/projects => generated 403 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:33.676984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45553e71-5f3d-4f8f-949d-e7a293cdd5ba None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:33.688307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45553e71-5f3d-4f8f-949d-e7a293cdd5ba None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:33.747047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b646d-9b03-4769-bd37-e0689cd7f632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:33.747654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b646d-9b03-4769-bd37-e0689cd7f632 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.747933 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b646d-9b03-4769-bd37-e0689cd7f632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.748200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b646d-9b03-4769-bd37-e0689cd7f632 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.748825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b646d-9b03-4769-bd37-e0689cd7f632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:33.750769 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b646d-9b03-4769-bd37-e0689cd7f632 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:33.759632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:33.775015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1155/2290] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:45:33 2022] POST /identity/v3/users => generated 308 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:33.776787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c938f-5a73-4642-b7eb-52e447573dcd admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:33.787207 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c938f-5a73-4642-b7eb-52e447573dcd admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:33.792507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2aad56-9a6f-4238-8841-c4c3da8025a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:33.793121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2aad56-9a6f-4238-8841-c4c3da8025a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.793389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2aad56-9a6f-4238-8841-c4c3da8025a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.793659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2aad56-9a6f-4238-8841-c4c3da8025a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.794286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2aad56-9a6f-4238-8841-c4c3da8025a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:33.796147 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2aad56-9a6f-4238-8841-c4c3da8025a5 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:33.806507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1136/2291] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:33.810691 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b646d-9b03-4769-bd37-e0689cd7f632 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:33.820807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b646d-9b03-4769-bd37-e0689cd7f632 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:33.825614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d0e53-37e7-4161-b7e4-92bb43111c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:33.826265 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d0e53-37e7-4161-b7e4-92bb43111c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.826544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d0e53-37e7-4161-b7e4-92bb43111c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.826813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d0e53-37e7-4161-b7e4-92bb43111c32 admin admin] PATH_INFO: `/v3/projects/e7bb33fca895478fbc107a359c053c17/users/ab564fb3e39c41c3a1a1dd6ed5ed1527/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.844301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d0e53-37e7-4161-b7e4-92bb43111c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bb33fca895478fbc107a359c053c17, user_id=ab564fb3e39c41c3a1a1dd6ed5ed1527, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:33.847076 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d0e53-37e7-4161-b7e4-92bb43111c32 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:33.864315 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1156/2292] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:33 2022] PUT /identity/v3/projects/e7bb33fca895478fbc107a359c053c17/users/ab564fb3e39c41c3a1a1dd6ed5ed1527/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:33.868342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2aad56-9a6f-4238-8841-c4c3da8025a5 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:33.878471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2aad56-9a6f-4238-8841-c4c3da8025a5 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:33.937061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f19dc-b9bb-4bcd-aba7-8ca7f7ad84a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:33.937703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f19dc-b9bb-4bcd-aba7-8ca7f7ad84a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.937985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f19dc-b9bb-4bcd-aba7-8ca7f7ad84a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.938349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f19dc-b9bb-4bcd-aba7-8ca7f7ad84a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.939143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f19dc-b9bb-4bcd-aba7-8ca7f7ad84a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:33.940866 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f19dc-b9bb-4bcd-aba7-8ca7f7ad84a1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:33.950701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1137/2293] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:33.955105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d0e53-37e7-4161-b7e4-92bb43111c32 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:33.965940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d0e53-37e7-4161-b7e4-92bb43111c32 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:33.971033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bcd29-d8ce-4693-97d2-c89f440767bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:33.971623 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bcd29-d8ce-4693-97d2-c89f440767bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.971901 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bcd29-d8ce-4693-97d2-c89f440767bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.972171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bcd29-d8ce-4693-97d2-c89f440767bc admin admin] PATH_INFO: `/v3/projects/e7bb33fca895478fbc107a359c053c17/users/ab564fb3e39c41c3a1a1dd6ed5ed1527/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:33.974127 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60bcd29-d8ce-4693-97d2-c89f440767bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bb33fca895478fbc107a359c053c17, user_id=ab564fb3e39c41c3a1a1dd6ed5ed1527, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:33.976722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60bcd29-d8ce-4693-97d2-c89f440767bc admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:33.987550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1157/2294] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:33 2022] PUT /identity/v3/projects/e7bb33fca895478fbc107a359c053c17/users/ab564fb3e39c41c3a1a1dd6ed5ed1527/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:33.997589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f2bc8-902a-472b-a934-8f673fd56408 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:33.997894 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f2bc8-902a-472b-a934-8f673fd56408 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:33.998164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f2bc8-902a-472b-a934-8f673fd56408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:34.030069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-210f2bc8-902a-472b-a934-8f673fd56408 None None] MFA Rules not processed for user `ab564fb3e39c41c3a1a1dd6ed5ed1527`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:34.031345 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-210f2bc8-902a-472b-a934-8f673fd56408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:34.101318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1138/2295] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:33 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:34.112671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bcd29-d8ce-4693-97d2-c89f440767bc admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:34.114414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c60bcd29-d8ce-4693-97d2-c89f440767bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:34.173631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bcd29-d8ce-4693-97d2-c89f440767bc admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:34.179369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97761d-1e55-4452-9d84-5746af971b0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:34.179981 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97761d-1e55-4452-9d84-5746af971b0b service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:34.180262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97761d-1e55-4452-9d84-5746af971b0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:34.180532 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97761d-1e55-4452-9d84-5746af971b0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:34.192664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97761d-1e55-4452-9d84-5746af971b0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:34.194126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97761d-1e55-4452-9d84-5746af971b0b service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:34.204217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1158/2296] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:34 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:34.355217 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f2bc8-902a-472b-a934-8f673fd56408 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:34.365199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:34.422690 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f2bc8-902a-472b-a934-8f673fd56408 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:34.427161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:34.427783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:34.428058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:34.428328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] PATH_INFO: `/v3/users/1d60f9a9799e443186ac551499c2805b` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:34.429480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d60f9a9799e443186ac551499c2805b)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:34.431374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:34.483581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d60f9a9799e443186ac551499c2805b'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:34.491014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d60f9a9799e443186ac551499c2805b'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:34.491730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d60f9a9799e443186ac551499c2805b'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:34.492331 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d60f9a9799e443186ac551499c2805b'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:34.502058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d60f9a9799e443186ac551499c2805b'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:34.530436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1139/2297] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:45:34 2022] DELETE /identity/v3/users/1d60f9a9799e443186ac551499c2805b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:34.535217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad48067-0762-4d23-a425-c6220d53ddb8 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:34.535502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad48067-0762-4d23-a425-c6220d53ddb8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:34.535769 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad48067-0762-4d23-a425-c6220d53ddb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:34.557754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-8ad48067-0762-4d23-a425-c6220d53ddb8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:34.559086 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8ad48067-0762-4d23-a425-c6220d53ddb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:34.627600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1159/2298] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:45:34.637887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:34.639395 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:34.702390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b9797-2441-47ba-b2c7-94bbaab0c898 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:34.706196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713f917-2e57-456d-82eb-47824ceaa0fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:34.706818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713f917-2e57-456d-82eb-47824ceaa0fd service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:34.707100 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713f917-2e57-456d-82eb-47824ceaa0fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:34.707376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713f917-2e57-456d-82eb-47824ceaa0fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:34.717730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713f917-2e57-456d-82eb-47824ceaa0fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:34.719207 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713f917-2e57-456d-82eb-47824ceaa0fd service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:34.729972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1140/2299] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:45:35.062980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad48067-0762-4d23-a425-c6220d53ddb8 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:35.064480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8ad48067-0762-4d23-a425-c6220d53ddb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:35.076870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad48067-0762-4d23-a425-c6220d53ddb8 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:35.083944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:35.084675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:35.084960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:35.085325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] PATH_INFO: `/v3/projects/1c568697ed76457e83d37e7ad9aee06a` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:35.086611 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c568697ed76457e83d37e7ad9aee06a)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:35.088811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:35.139642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c568697ed76457e83d37e7ad9aee06a', 'request_id': 'req-7b31724a-d332-44fa-b37d-053ab9ea37f1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.141657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:35.147320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c568697ed76457e83d37e7ad9aee06a', 'request_id': 'req-7b31724a-d332-44fa-b37d-053ab9ea37f1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.154776 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c568697ed76457e83d37e7ad9aee06a', 'request_id': 'req-7b31724a-d332-44fa-b37d-053ab9ea37f1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.169318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] The token cache is being invalidate because project 1c568697ed76457e83d37e7ad9aee06a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:45:35.169611 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.170252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.172203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1160/2300] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:45:35 2022] DELETE /identity/v3/projects/1c568697ed76457e83d37e7ad9aee06a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:35.176340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713f917-2e57-456d-82eb-47824ceaa0fd service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:35.187182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713f917-2e57-456d-82eb-47824ceaa0fd service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:35.224281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:35.257371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2281c6-553d-4681-8286-c2dd59aaf65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:35.258075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2281c6-553d-4681-8286-c2dd59aaf65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:35.258367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2281c6-553d-4681-8286-c2dd59aaf65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:35.258647 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2281c6-553d-4681-8286-c2dd59aaf65a admin admin] PATH_INFO: `/v3/users/ab564fb3e39c41c3a1a1dd6ed5ed1527` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:35.259799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2281c6-553d-4681-8286-c2dd59aaf65a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab564fb3e39c41c3a1a1dd6ed5ed1527)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:35.261859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2281c6-553d-4681-8286-c2dd59aaf65a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:35.313679 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8f2281c6-553d-4681-8286-c2dd59aaf65a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab564fb3e39c41c3a1a1dd6ed5ed1527'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.320745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8f2281c6-553d-4681-8286-c2dd59aaf65a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab564fb3e39c41c3a1a1dd6ed5ed1527'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.321419 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8f2281c6-553d-4681-8286-c2dd59aaf65a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab564fb3e39c41c3a1a1dd6ed5ed1527'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.322015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8f2281c6-553d-4681-8286-c2dd59aaf65a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab564fb3e39c41c3a1a1dd6ed5ed1527'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.331152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8f2281c6-553d-4681-8286-c2dd59aaf65a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab564fb3e39c41c3a1a1dd6ed5ed1527'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.349758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1141/2301] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:45:35 2022] DELETE /identity/v3/users/ab564fb3e39c41c3a1a1dd6ed5ed1527 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:35.774909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:35.777372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:35.790728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31724a-d332-44fa-b37d-053ab9ea37f1 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:35.856608 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330eb5a-099d-4a71-aa24-2b889895782d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:35.857284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330eb5a-099d-4a71-aa24-2b889895782d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:35.857568 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330eb5a-099d-4a71-aa24-2b889895782d admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:35.857844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330eb5a-099d-4a71-aa24-2b889895782d admin admin] PATH_INFO: `/v3/projects/e7bb33fca895478fbc107a359c053c17` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:35.859056 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330eb5a-099d-4a71-aa24-2b889895782d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7bb33fca895478fbc107a359c053c17)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:35.861297 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330eb5a-099d-4a71-aa24-2b889895782d admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:35.910955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-a330eb5a-099d-4a71-aa24-2b889895782d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7bb33fca895478fbc107a359c053c17', 'request_id': 'req-a330eb5a-099d-4a71-aa24-2b889895782d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.916675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-a330eb5a-099d-4a71-aa24-2b889895782d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7bb33fca895478fbc107a359c053c17', 'request_id': 'req-a330eb5a-099d-4a71-aa24-2b889895782d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.923481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-a330eb5a-099d-4a71-aa24-2b889895782d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7bb33fca895478fbc107a359c053c17', 'request_id': 'req-a330eb5a-099d-4a71-aa24-2b889895782d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.936582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-a330eb5a-099d-4a71-aa24-2b889895782d admin admin] The token cache is being invalidate because project e7bb33fca895478fbc107a359c053c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:45:35.936874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-a330eb5a-099d-4a71-aa24-2b889895782d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.937550 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-a330eb5a-099d-4a71-aa24-2b889895782d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:35.939054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1161/2302] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:45:35 2022] DELETE /identity/v3/projects/e7bb33fca895478fbc107a359c053c17 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:35.943947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2b027-9ce7-4504-8697-685060e26981 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:35.944444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2b027-9ce7-4504-8697-685060e26981 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:35.944550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2b027-9ce7-4504-8697-685060e26981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:35.965807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-35d2b027-9ce7-4504-8697-685060e26981 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:35.967013 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-35d2b027-9ce7-4504-8697-685060e26981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:36.034322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1142/2303] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:36.043389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb9a14-19dc-4a8a-93c0-f9ce3b13d76a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.043673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb9a14-19dc-4a8a-93c0-f9ce3b13d76a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.043949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb9a14-19dc-4a8a-93c0-f9ce3b13d76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.067081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-ccfb9a14-19dc-4a8a-93c0-f9ce3b13d76a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:36.068318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ccfb9a14-19dc-4a8a-93c0-f9ce3b13d76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:36.077681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1162/2304] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:36.081926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2b027-9ce7-4504-8697-685060e26981 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:36.094243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2b027-9ce7-4504-8697-685060e26981 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:36.099213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5023b5-4c3b-4975-b8cd-cc9b96bca6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:36.099859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5023b5-4c3b-4975-b8cd-cc9b96bca6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.100141 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5023b5-4c3b-4975-b8cd-cc9b96bca6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.100418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5023b5-4c3b-4975-b8cd-cc9b96bca6f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.101067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5023b5-4c3b-4975-b8cd-cc9b96bca6f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:45:36.101405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5023b5-4c3b-4975-b8cd-cc9b96bca6f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:36.103044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5023b5-4c3b-4975-b8cd-cc9b96bca6f3 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:36.115589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1143/2305] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:45:36.120812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb9a14-19dc-4a8a-93c0-f9ce3b13d76a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:36.130733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb9a14-19dc-4a8a-93c0-f9ce3b13d76a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:36.136838 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9032f2-8fae-41f9-8dcf-c0a631bb34ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:36.137516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9032f2-8fae-41f9-8dcf-c0a631bb34ef admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.137803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9032f2-8fae-41f9-8dcf-c0a631bb34ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.138083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9032f2-8fae-41f9-8dcf-c0a631bb34ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.138739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9032f2-8fae-41f9-8dcf-c0a631bb34ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:36.140676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9032f2-8fae-41f9-8dcf-c0a631bb34ef admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:36.181529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1163/2306] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:36 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:36.186304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5023b5-4c3b-4975-b8cd-cc9b96bca6f3 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:36.198795 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5023b5-4c3b-4975-b8cd-cc9b96bca6f3 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:36.240847 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:36.279507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef8420-8275-40b7-bfd5-3573cd37830a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:36.280151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef8420-8275-40b7-bfd5-3573cd37830a admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.280752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef8420-8275-40b7-bfd5-3573cd37830a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.280752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef8420-8275-40b7-bfd5-3573cd37830a admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.281363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef8420-8275-40b7-bfd5-3573cd37830a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:36.283299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef8420-8275-40b7-bfd5-3573cd37830a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:36.309034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1144/2307] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:45:36 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:45:36.313240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9032f2-8fae-41f9-8dcf-c0a631bb34ef admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:36.324875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9032f2-8fae-41f9-8dcf-c0a631bb34ef admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:36.330133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203b850-bcf8-4d76-ab2a-a1c10147e91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:36.330754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203b850-bcf8-4d76-ab2a-a1c10147e91a admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.331038 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203b850-bcf8-4d76-ab2a-a1c10147e91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.331315 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203b850-bcf8-4d76-ab2a-a1c10147e91a admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.331992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203b850-bcf8-4d76-ab2a-a1c10147e91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:36.333629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203b850-bcf8-4d76-ab2a-a1c10147e91a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:36.346661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1164/2308] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:36.350912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef8420-8275-40b7-bfd5-3573cd37830a admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:36.363969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef8420-8275-40b7-bfd5-3573cd37830a admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:36.369153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81249972-fdae-441d-a85c-b24ab0a31793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:36.370690 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81249972-fdae-441d-a85c-b24ab0a31793 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.371011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81249972-fdae-441d-a85c-b24ab0a31793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.371396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81249972-fdae-441d-a85c-b24ab0a31793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.372159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81249972-fdae-441d-a85c-b24ab0a31793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:36.374001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81249972-fdae-441d-a85c-b24ab0a31793 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:36.388322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1145/2309] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:36.389950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203b850-bcf8-4d76-ab2a-a1c10147e91a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:36.400143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203b850-bcf8-4d76-ab2a-a1c10147e91a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:36.404923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33161b08-c37f-43a8-b6c8-5181f05bf016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:36.405712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33161b08-c37f-43a8-b6c8-5181f05bf016 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.406722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33161b08-c37f-43a8-b6c8-5181f05bf016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.406722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33161b08-c37f-43a8-b6c8-5181f05bf016 admin admin] PATH_INFO: `/v3/projects/217e578e73a8435cbc60b0472114d3bf/users/c7783a376b14497097ddd8b0820c23d4/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.426325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33161b08-c37f-43a8-b6c8-5181f05bf016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=217e578e73a8435cbc60b0472114d3bf, user_id=c7783a376b14497097ddd8b0820c23d4, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:36.429200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33161b08-c37f-43a8-b6c8-5181f05bf016 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:36.445197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1165/2310] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:36 2022] PUT /identity/v3/projects/217e578e73a8435cbc60b0472114d3bf/users/c7783a376b14497097ddd8b0820c23d4/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:45:36.456412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024200e-50a8-4c38-9f4f-321424c57077 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.457006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024200e-50a8-4c38-9f4f-321424c57077 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.457503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024200e-50a8-4c38-9f4f-321424c57077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.490077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-7024200e-50a8-4c38-9f4f-321424c57077 None None] MFA Rules not processed for user `c7783a376b14497097ddd8b0820c23d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:36.491405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7024200e-50a8-4c38-9f4f-321424c57077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:36.573614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1146/2311] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:45:36.578920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33161b08-c37f-43a8-b6c8-5181f05bf016 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:36.591513 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:36.656661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33161b08-c37f-43a8-b6c8-5181f05bf016 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:36.661836 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277a512-c8e8-450e-b0a8-d41d60a6ac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:36.662537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277a512-c8e8-450e-b0a8-d41d60a6ac6a admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.662886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277a512-c8e8-450e-b0a8-d41d60a6ac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.663258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277a512-c8e8-450e-b0a8-d41d60a6ac6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.664491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277a512-c8e8-450e-b0a8-d41d60a6ac6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:36.665950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277a512-c8e8-450e-b0a8-d41d60a6ac6a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:36.699271 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1166/2312] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:36 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:36.700581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024200e-50a8-4c38-9f4f-321424c57077 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:36.714447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024200e-50a8-4c38-9f4f-321424c57077 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:36.776296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cb8b3-c5b8-4c75-a303-49fe706d4485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:36.776686 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cb8b3-c5b8-4c75-a303-49fe706d4485 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.776965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cb8b3-c5b8-4c75-a303-49fe706d4485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.777276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cb8b3-c5b8-4c75-a303-49fe706d4485 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.778143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cb8b3-c5b8-4c75-a303-49fe706d4485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:36.780065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cb8b3-c5b8-4c75-a303-49fe706d4485 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:36.802678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1147/2313] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:45:36 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:36.806791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277a512-c8e8-450e-b0a8-d41d60a6ac6a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:36.817207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277a512-c8e8-450e-b0a8-d41d60a6ac6a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:36.822444 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aad10c-fa1f-41d1-91ff-6515fdd8f0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:36.823059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aad10c-fa1f-41d1-91ff-6515fdd8f0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.823344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aad10c-fa1f-41d1-91ff-6515fdd8f0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.823678 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aad10c-fa1f-41d1-91ff-6515fdd8f0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.824492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aad10c-fa1f-41d1-91ff-6515fdd8f0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:36.826263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aad10c-fa1f-41d1-91ff-6515fdd8f0f3 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:36.840833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1167/2314] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:45:36.845640 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cb8b3-c5b8-4c75-a303-49fe706d4485 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:36.856237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cb8b3-c5b8-4c75-a303-49fe706d4485 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:36.861075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7eedf8-fc41-4005-9062-306554a33780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:36.861822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7eedf8-fc41-4005-9062-306554a33780 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.862110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7eedf8-fc41-4005-9062-306554a33780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.862380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7eedf8-fc41-4005-9062-306554a33780 admin admin] PATH_INFO: `/v3/projects/f4c82a7b844a4a5eb2f4616ec197adbb/users/454e3c092f2a45ee864fb8fecfa2cc44/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.879156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7eedf8-fc41-4005-9062-306554a33780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c82a7b844a4a5eb2f4616ec197adbb, user_id=454e3c092f2a45ee864fb8fecfa2cc44, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:36.881924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7eedf8-fc41-4005-9062-306554a33780 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:36.894892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1148/2315] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:36 2022] PUT /identity/v3/projects/f4c82a7b844a4a5eb2f4616ec197adbb/users/454e3c092f2a45ee864fb8fecfa2cc44/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:45:36.898995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aad10c-fa1f-41d1-91ff-6515fdd8f0f3 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:36.910641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aad10c-fa1f-41d1-91ff-6515fdd8f0f3 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:36.972908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615fe0a-0acc-4a14-afec-a86f3b04ae80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:36.973572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615fe0a-0acc-4a14-afec-a86f3b04ae80 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:36.973855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615fe0a-0acc-4a14-afec-a86f3b04ae80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:36.974130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615fe0a-0acc-4a14-afec-a86f3b04ae80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:36.974813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615fe0a-0acc-4a14-afec-a86f3b04ae80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:36.976426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615fe0a-0acc-4a14-afec-a86f3b04ae80 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:36.987773 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1168/2316] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:36.992167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7eedf8-fc41-4005-9062-306554a33780 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:37.003229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7eedf8-fc41-4005-9062-306554a33780 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:37.008038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2645c6-5231-45c1-bdd9-1276ec0ce3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:37.008825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2645c6-5231-45c1-bdd9-1276ec0ce3eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:37.009138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2645c6-5231-45c1-bdd9-1276ec0ce3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:37.009423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2645c6-5231-45c1-bdd9-1276ec0ce3eb admin admin] PATH_INFO: `/v3/projects/f4c82a7b844a4a5eb2f4616ec197adbb/users/454e3c092f2a45ee864fb8fecfa2cc44/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:37.012098 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2645c6-5231-45c1-bdd9-1276ec0ce3eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c82a7b844a4a5eb2f4616ec197adbb, user_id=454e3c092f2a45ee864fb8fecfa2cc44, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:37.014826 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2645c6-5231-45c1-bdd9-1276ec0ce3eb admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:37.020936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:37.028544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1149/2317] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:36 2022] PUT /identity/v3/projects/f4c82a7b844a4a5eb2f4616ec197adbb/users/454e3c092f2a45ee864fb8fecfa2cc44/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:37.038684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509d6db-649f-48a0-b5f2-1ce88b39d6fa None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:37.039206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509d6db-649f-48a0-b5f2-1ce88b39d6fa None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:37.039488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509d6db-649f-48a0-b5f2-1ce88b39d6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:37.070765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-0509d6db-649f-48a0-b5f2-1ce88b39d6fa None None] MFA Rules not processed for user `454e3c092f2a45ee864fb8fecfa2cc44`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:37.072013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-0509d6db-649f-48a0-b5f2-1ce88b39d6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:37.145453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1169/2318] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:37.158768 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2645c6-5231-45c1-bdd9-1276ec0ce3eb admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:37.160483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ad2645c6-5231-45c1-bdd9-1276ec0ce3eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:37.221955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2645c6-5231-45c1-bdd9-1276ec0ce3eb admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:37.225903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd51f0-f401-45d4-bc5e-a7725199948b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:37.226509 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd51f0-f401-45d4-bc5e-a7725199948b service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:37.226791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd51f0-f401-45d4-bc5e-a7725199948b service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:37.227066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd51f0-f401-45d4-bc5e-a7725199948b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:37.236766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd51f0-f401-45d4-bc5e-a7725199948b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:37.238559 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd51f0-f401-45d4-bc5e-a7725199948b service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:37.251823 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1150/2319] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:37.567914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509d6db-649f-48a0-b5f2-1ce88b39d6fa None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:37.572362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:37.579231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509d6db-649f-48a0-b5f2-1ce88b39d6fa None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:37.584484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c4065-cd08-4ce4-b5c1-3170a7e92cc9 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454e3c092f2a45ee864fb8fecfa2cc44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4c82a7b844a4a5eb2f4616ec197adbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:37.585119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c4065-cd08-4ce4-b5c1-3170a7e92cc9 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:37.585405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c4065-cd08-4ce4-b5c1-3170a7e92cc9 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:37.585682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c4065-cd08-4ce4-b5c1-3170a7e92cc9 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:37.586335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c4065-cd08-4ce4-b5c1-3170a7e92cc9 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:45:37.586637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c4065-cd08-4ce4-b5c1-3170a7e92cc9 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:37.588224 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c4065-cd08-4ce4-b5c1-3170a7e92cc9 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:37.600060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1170/2320] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:45:37.604372 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd51f0-f401-45d4-bc5e-a7725199948b service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:37.613590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd51f0-f401-45d4-bc5e-a7725199948b service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:37.618215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83495da-9f99-4140-a25e-7c80d335dbd6 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454e3c092f2a45ee864fb8fecfa2cc44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4c82a7b844a4a5eb2f4616ec197adbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:37.618816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83495da-9f99-4140-a25e-7c80d335dbd6 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:37.619100 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83495da-9f99-4140-a25e-7c80d335dbd6 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:37.619880 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83495da-9f99-4140-a25e-7c80d335dbd6 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:37.620544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83495da-9f99-4140-a25e-7c80d335dbd6 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:37.622484 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83495da-9f99-4140-a25e-7c80d335dbd6 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:37.649351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1151/2321] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:37 2022] POST /identity/v3/projects => generated 354 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:37.653469 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c4065-cd08-4ce4-b5c1-3170a7e92cc9 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:37.663425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c4065-cd08-4ce4-b5c1-3170a7e92cc9 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:37.720193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8b55e-63c7-466a-98fd-c9cfb12c3bfd tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454e3c092f2a45ee864fb8fecfa2cc44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4c82a7b844a4a5eb2f4616ec197adbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:37.720799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8b55e-63c7-466a-98fd-c9cfb12c3bfd tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:37.721084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8b55e-63c7-466a-98fd-c9cfb12c3bfd tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:37.721393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8b55e-63c7-466a-98fd-c9cfb12c3bfd tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:37.722034 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8b55e-63c7-466a-98fd-c9cfb12c3bfd tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:45:37.722351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8b55e-63c7-466a-98fd-c9cfb12c3bfd tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:37.723926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8b55e-63c7-466a-98fd-c9cfb12c3bfd tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:37.735546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1171/2322] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:45:37.773037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83495da-9f99-4140-a25e-7c80d335dbd6 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:37.783630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83495da-9f99-4140-a25e-7c80d335dbd6 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:37.788362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '454e3c092f2a45ee864fb8fecfa2cc44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f4c82a7b844a4a5eb2f4616ec197adbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:37.788929 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:37.789250 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:37.789532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] PATH_INFO: `/v3/projects/7825d75aa86e4ef6a01c3b28ecba0994` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:37.790653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7825d75aa86e4ef6a01c3b28ecba0994)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:37.792574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:37.841866 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7825d75aa86e4ef6a01c3b28ecba0994', 'request_id': 'req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:37.847450 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7825d75aa86e4ef6a01c3b28ecba0994', 'request_id': 'req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:37.854466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7825d75aa86e4ef6a01c3b28ecba0994', 'request_id': 'req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:37.867526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] The token cache is being invalidate because project 7825d75aa86e4ef6a01c3b28ecba0994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:45:37.867815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:37.868467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ffc935ea-ec5c-48fc-ba3f-a29fd360cc83 tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:37.870011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1152/2323] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:45:37 2022] DELETE /identity/v3/projects/7825d75aa86e4ef6a01c3b28ecba0994 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:38.060099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8b55e-63c7-466a-98fd-c9cfb12c3bfd tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:38.062029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-55f8b55e-63c7-466a-98fd-c9cfb12c3bfd tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:38.126053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8b55e-63c7-466a-98fd-c9cfb12c3bfd tempest-VolumeQuotaClassesTest-428258576 tempest-VolumeQuotaClassesTest-428258576-project-admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:38.182524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:38.183715 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:38.183996 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:38.184269 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] PATH_INFO: `/v3/users/c7783a376b14497097ddd8b0820c23d4` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:38.185544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7783a376b14497097ddd8b0820c23d4)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:38.189665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:38.230512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:38.244317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7783a376b14497097ddd8b0820c23d4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.256256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7783a376b14497097ddd8b0820c23d4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.256919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7783a376b14497097ddd8b0820c23d4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.264397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7783a376b14497097ddd8b0820c23d4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.265115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7783a376b14497097ddd8b0820c23d4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.280634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1172/2324] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:45:38 2022] DELETE /identity/v3/users/c7783a376b14497097ddd8b0820c23d4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:45:38.285219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c915c5-6d75-499a-87d0-a303664d4575 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:38.285746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c915c5-6d75-499a-87d0-a303664d4575 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:38.285853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c915c5-6d75-499a-87d0-a303664d4575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:38.305704 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-f7c915c5-6d75-499a-87d0-a303664d4575 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:38.306882 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f7c915c5-6d75-499a-87d0-a303664d4575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:38.364263 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1153/2325] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:45:38.373945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:38.375341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:38.433458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbe18f-1776-46dd-a202-ff3d540873f2 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:38.437037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a943718-5777-43fa-9ba9-779f8d8704a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:38.437664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a943718-5777-43fa-9ba9-779f8d8704a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:38.437944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a943718-5777-43fa-9ba9-779f8d8704a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:38.438215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a943718-5777-43fa-9ba9-779f8d8704a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:38.447766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a943718-5777-43fa-9ba9-779f8d8704a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:38.449204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a943718-5777-43fa-9ba9-779f8d8704a4 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:38.458644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1173/2326] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:38.717896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c915c5-6d75-499a-87d0-a303664d4575 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:38.719446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f7c915c5-6d75-499a-87d0-a303664d4575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:38.726811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:38.734122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c915c5-6d75-499a-87d0-a303664d4575 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:38.740973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:38.741723 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:38.742076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:38.742425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] PATH_INFO: `/v3/projects/217e578e73a8435cbc60b0472114d3bf` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:38.744708 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=217e578e73a8435cbc60b0472114d3bf)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:38.746770 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:38.795131 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '217e578e73a8435cbc60b0472114d3bf', 'request_id': 'req-ea635597-70d3-4aeb-bcac-756bbd730710'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.800857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '217e578e73a8435cbc60b0472114d3bf', 'request_id': 'req-ea635597-70d3-4aeb-bcac-756bbd730710'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.808214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '217e578e73a8435cbc60b0472114d3bf', 'request_id': 'req-ea635597-70d3-4aeb-bcac-756bbd730710'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.819466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] The token cache is being invalidate because project 217e578e73a8435cbc60b0472114d3bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:45:38.819756 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.820377 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.821882 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1154/2327] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:45:38 2022] DELETE /identity/v3/projects/217e578e73a8435cbc60b0472114d3bf => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:38.826507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a943718-5777-43fa-9ba9-779f8d8704a4 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:38.836120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a943718-5777-43fa-9ba9-779f8d8704a4 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:38.893971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4b00c-b8a0-4672-83f9-03857637ff96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:38.894516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4b00c-b8a0-4672-83f9-03857637ff96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:38.894793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4b00c-b8a0-4672-83f9-03857637ff96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:38.895067 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4b00c-b8a0-4672-83f9-03857637ff96 admin admin] PATH_INFO: `/v3/users/454e3c092f2a45ee864fb8fecfa2cc44` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:38.896173 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4b00c-b8a0-4672-83f9-03857637ff96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=454e3c092f2a45ee864fb8fecfa2cc44)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:38.899671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4b00c-b8a0-4672-83f9-03857637ff96 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:38.946931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bcb4b00c-b8a0-4672-83f9-03857637ff96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454e3c092f2a45ee864fb8fecfa2cc44'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.952657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:38.960555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bcb4b00c-b8a0-4672-83f9-03857637ff96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454e3c092f2a45ee864fb8fecfa2cc44'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.961399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bcb4b00c-b8a0-4672-83f9-03857637ff96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454e3c092f2a45ee864fb8fecfa2cc44'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.969334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bcb4b00c-b8a0-4672-83f9-03857637ff96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454e3c092f2a45ee864fb8fecfa2cc44'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.970044 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-bcb4b00c-b8a0-4672-83f9-03857637ff96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454e3c092f2a45ee864fb8fecfa2cc44'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:38.982490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1174/2328] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:45:38 2022] DELETE /identity/v3/users/454e3c092f2a45ee864fb8fecfa2cc44 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:39.254695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:39.256140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:39.269953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea635597-70d3-4aeb-bcac-756bbd730710 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:39.327575 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99965c81-92d9-46fe-af8c-b49971727833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:39.328249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99965c81-92d9-46fe-af8c-b49971727833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:39.328466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99965c81-92d9-46fe-af8c-b49971727833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:39.328733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99965c81-92d9-46fe-af8c-b49971727833 admin admin] PATH_INFO: `/v3/projects/f4c82a7b844a4a5eb2f4616ec197adbb` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:39.329911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99965c81-92d9-46fe-af8c-b49971727833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c82a7b844a4a5eb2f4616ec197adbb)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:39.331808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99965c81-92d9-46fe-af8c-b49971727833 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:39.378147 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-99965c81-92d9-46fe-af8c-b49971727833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c82a7b844a4a5eb2f4616ec197adbb', 'request_id': 'req-99965c81-92d9-46fe-af8c-b49971727833'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:39.383580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-99965c81-92d9-46fe-af8c-b49971727833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c82a7b844a4a5eb2f4616ec197adbb', 'request_id': 'req-99965c81-92d9-46fe-af8c-b49971727833'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:39.391769 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-99965c81-92d9-46fe-af8c-b49971727833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c82a7b844a4a5eb2f4616ec197adbb', 'request_id': 'req-99965c81-92d9-46fe-af8c-b49971727833'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:39.402922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-99965c81-92d9-46fe-af8c-b49971727833 admin admin] The token cache is being invalidate because project f4c82a7b844a4a5eb2f4616ec197adbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:45:39.403241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-99965c81-92d9-46fe-af8c-b49971727833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:39.403894 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-99965c81-92d9-46fe-af8c-b49971727833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:39.405441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1155/2329] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:45:39 2022] DELETE /identity/v3/projects/f4c82a7b844a4a5eb2f4616ec197adbb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:39.410417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88685d6a-b363-435d-89ce-d28bf476156b None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:39.410888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88685d6a-b363-435d-89ce-d28bf476156b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:39.410997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88685d6a-b363-435d-89ce-d28bf476156b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:39.431081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-88685d6a-b363-435d-89ce-d28bf476156b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:39.432222 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-88685d6a-b363-435d-89ce-d28bf476156b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:39.490856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1175/2330] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:39.499577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bc12a-3253-4c3f-9eee-1083a08c69b2 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:39.499857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bc12a-3253-4c3f-9eee-1083a08c69b2 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:39.500129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bc12a-3253-4c3f-9eee-1083a08c69b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:39.520787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-0f8bc12a-3253-4c3f-9eee-1083a08c69b2 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:39.521758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-0f8bc12a-3253-4c3f-9eee-1083a08c69b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:39.530441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1156/2331] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:39.534651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88685d6a-b363-435d-89ce-d28bf476156b None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:39.544390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88685d6a-b363-435d-89ce-d28bf476156b None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:39.549373 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76373a-e50a-48f0-846d-5ac03e359e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:39.549967 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76373a-e50a-48f0-846d-5ac03e359e24 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:39.550292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76373a-e50a-48f0-846d-5ac03e359e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:39.550524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76373a-e50a-48f0-846d-5ac03e359e24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:39.551172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76373a-e50a-48f0-846d-5ac03e359e24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:45:39.551476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76373a-e50a-48f0-846d-5ac03e359e24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:39.553043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76373a-e50a-48f0-846d-5ac03e359e24 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:39.564629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1176/2332] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:45:39.568923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8bc12a-3253-4c3f-9eee-1083a08c69b2 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:39.578534 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8bc12a-3253-4c3f-9eee-1083a08c69b2 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:39.583119 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548b5f9-3022-4f40-978f-1fb5467c6d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:39.583718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548b5f9-3022-4f40-978f-1fb5467c6d6a admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:39.583992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548b5f9-3022-4f40-978f-1fb5467c6d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:39.584263 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548b5f9-3022-4f40-978f-1fb5467c6d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:39.584903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548b5f9-3022-4f40-978f-1fb5467c6d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:39.586818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548b5f9-3022-4f40-978f-1fb5467c6d6a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:39.592966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:39.616380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1157/2333] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:39 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:39.620618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76373a-e50a-48f0-846d-5ac03e359e24 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:39.630714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76373a-e50a-48f0-846d-5ac03e359e24 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:39.687324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522eb50-741a-45d6-ac6a-e036ebae316b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:39.688372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522eb50-741a-45d6-ac6a-e036ebae316b admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:39.688372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522eb50-741a-45d6-ac6a-e036ebae316b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:39.688372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522eb50-741a-45d6-ac6a-e036ebae316b admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:39.688979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522eb50-741a-45d6-ac6a-e036ebae316b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:39.690937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522eb50-741a-45d6-ac6a-e036ebae316b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:39.714987 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1177/2334] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:45:39 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:39.719162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548b5f9-3022-4f40-978f-1fb5467c6d6a admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:39.729639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548b5f9-3022-4f40-978f-1fb5467c6d6a admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:39.734273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eb3eb-cc1f-4293-8cda-15243625ec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:39.734872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eb3eb-cc1f-4293-8cda-15243625ec4b admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:39.735152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eb3eb-cc1f-4293-8cda-15243625ec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:39.735427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eb3eb-cc1f-4293-8cda-15243625ec4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:39.736090 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42eb3eb-cc1f-4293-8cda-15243625ec4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:39.737708 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42eb3eb-cc1f-4293-8cda-15243625ec4b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:39.748078 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1158/2335] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:39.751999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522eb50-741a-45d6-ac6a-e036ebae316b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:39.762859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522eb50-741a-45d6-ac6a-e036ebae316b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:39.767514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f6742-bbf1-4825-8210-3fbc4ac7f862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:39.768109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f6742-bbf1-4825-8210-3fbc4ac7f862 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:39.768392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f6742-bbf1-4825-8210-3fbc4ac7f862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:39.768669 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f6742-bbf1-4825-8210-3fbc4ac7f862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:39.769344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f6742-bbf1-4825-8210-3fbc4ac7f862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:39.770924 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f6742-bbf1-4825-8210-3fbc4ac7f862 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:39.781677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1178/2336] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:39.785894 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eb3eb-cc1f-4293-8cda-15243625ec4b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:39.796219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eb3eb-cc1f-4293-8cda-15243625ec4b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:39.800749 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577303b-2230-47fe-8515-cec4edaa8d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:39.801417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577303b-2230-47fe-8515-cec4edaa8d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:39.801693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577303b-2230-47fe-8515-cec4edaa8d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:39.801960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577303b-2230-47fe-8515-cec4edaa8d99 admin admin] PATH_INFO: `/v3/projects/74b33b34704e47baa300f1f8d68843ed/users/e67b70ac47814299a0f50a2c2b27053d/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:39.817209 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577303b-2230-47fe-8515-cec4edaa8d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b33b34704e47baa300f1f8d68843ed, user_id=e67b70ac47814299a0f50a2c2b27053d, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:39.819863 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577303b-2230-47fe-8515-cec4edaa8d99 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:39.833744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1159/2337] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:39 2022] PUT /identity/v3/projects/74b33b34704e47baa300f1f8d68843ed/users/e67b70ac47814299a0f50a2c2b27053d/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:39.843989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de669d-71ad-47bc-a67d-43ddd75de744 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:39.844276 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de669d-71ad-47bc-a67d-43ddd75de744 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:39.844546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de669d-71ad-47bc-a67d-43ddd75de744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:39.872174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-d8de669d-71ad-47bc-a67d-43ddd75de744 None None] MFA Rules not processed for user `e67b70ac47814299a0f50a2c2b27053d`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:39.873405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d8de669d-71ad-47bc-a67d-43ddd75de744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:39.899336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:39.940586 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1179/2338] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:39.945009 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577303b-2230-47fe-8515-cec4edaa8d99 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:40.012963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577303b-2230-47fe-8515-cec4edaa8d99 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:40.017833 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cb211-371a-4e93-9e23-e5a0e14455ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:40.018473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb211-371a-4e93-9e23-e5a0e14455ff admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:40.018714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb211-371a-4e93-9e23-e5a0e14455ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:40.018989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb211-371a-4e93-9e23-e5a0e14455ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:40.019624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cb211-371a-4e93-9e23-e5a0e14455ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:40.021610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cb211-371a-4e93-9e23-e5a0e14455ff admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:40.048118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1160/2339] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:39 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:40.052140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de669d-71ad-47bc-a67d-43ddd75de744 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:40.064013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de669d-71ad-47bc-a67d-43ddd75de744 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:40.121762 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d5aad-9a28-44a2-bad0-0c95fcec3754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:40.122371 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d5aad-9a28-44a2-bad0-0c95fcec3754 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:40.122649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d5aad-9a28-44a2-bad0-0c95fcec3754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:40.122922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d5aad-9a28-44a2-bad0-0c95fcec3754 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:40.123560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d5aad-9a28-44a2-bad0-0c95fcec3754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:40.125518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d5aad-9a28-44a2-bad0-0c95fcec3754 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:40.146255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1180/2340] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:45:40 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:45:40.150369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cb211-371a-4e93-9e23-e5a0e14455ff admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:40.160725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cb211-371a-4e93-9e23-e5a0e14455ff admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:40.165660 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4061db-c256-4d55-95fd-e9db88901b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:40.166254 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4061db-c256-4d55-95fd-e9db88901b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:40.166535 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4061db-c256-4d55-95fd-e9db88901b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:40.166803 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4061db-c256-4d55-95fd-e9db88901b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:40.167943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4061db-c256-4d55-95fd-e9db88901b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:40.169566 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4061db-c256-4d55-95fd-e9db88901b84 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:40.180337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1161/2341] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:45:40.185252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d5aad-9a28-44a2-bad0-0c95fcec3754 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:40.195269 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d5aad-9a28-44a2-bad0-0c95fcec3754 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:40.199793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d8fac-16ee-4824-b376-d4ca9c8a3b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:40.200436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d8fac-16ee-4824-b376-d4ca9c8a3b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:40.200723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d8fac-16ee-4824-b376-d4ca9c8a3b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:40.200998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d8fac-16ee-4824-b376-d4ca9c8a3b05 admin admin] PATH_INFO: `/v3/projects/07297b75cb704e16840cf1f66e329f82/users/e91060c07dd14c0abfdd4c3fd0668e2b/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:40.216167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d8fac-16ee-4824-b376-d4ca9c8a3b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07297b75cb704e16840cf1f66e329f82, user_id=e91060c07dd14c0abfdd4c3fd0668e2b, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:40.219337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d8fac-16ee-4824-b376-d4ca9c8a3b05 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:40.233651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1181/2342] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:40 2022] PUT /identity/v3/projects/07297b75cb704e16840cf1f66e329f82/users/e91060c07dd14c0abfdd4c3fd0668e2b/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:40.237554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4061db-c256-4d55-95fd-e9db88901b84 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:40.247962 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4061db-c256-4d55-95fd-e9db88901b84 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:40.304313 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9b78c-5e4b-4d87-8c75-965640a966c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:40.304922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9b78c-5e4b-4d87-8c75-965640a966c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:40.305239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9b78c-5e4b-4d87-8c75-965640a966c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:40.305518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9b78c-5e4b-4d87-8c75-965640a966c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:40.306152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9b78c-5e4b-4d87-8c75-965640a966c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:40.308358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9b78c-5e4b-4d87-8c75-965640a966c0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:40.318111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1162/2343] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:40.322387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d8fac-16ee-4824-b376-d4ca9c8a3b05 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:40.332708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d8fac-16ee-4824-b376-d4ca9c8a3b05 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:40.337362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a5914-2bc6-4018-b638-fbb37d2ea9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:40.338004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a5914-2bc6-4018-b638-fbb37d2ea9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:40.338284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a5914-2bc6-4018-b638-fbb37d2ea9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:40.338557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a5914-2bc6-4018-b638-fbb37d2ea9f3 admin admin] PATH_INFO: `/v3/projects/07297b75cb704e16840cf1f66e329f82/users/e91060c07dd14c0abfdd4c3fd0668e2b/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:40.340493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a5914-2bc6-4018-b638-fbb37d2ea9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07297b75cb704e16840cf1f66e329f82, user_id=e91060c07dd14c0abfdd4c3fd0668e2b, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:40.343167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a5914-2bc6-4018-b638-fbb37d2ea9f3 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:40.353177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1182/2344] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:40 2022] PUT /identity/v3/projects/07297b75cb704e16840cf1f66e329f82/users/e91060c07dd14c0abfdd4c3fd0668e2b/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:40.364657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753b912-b28c-4c68-90ea-ad154a73ec69 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:40.365194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753b912-b28c-4c68-90ea-ad154a73ec69 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:40.365301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753b912-b28c-4c68-90ea-ad154a73ec69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:40.394769 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-6753b912-b28c-4c68-90ea-ad154a73ec69 None None] MFA Rules not processed for user `e91060c07dd14c0abfdd4c3fd0668e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:40.395991 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6753b912-b28c-4c68-90ea-ad154a73ec69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:40.405947 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:40.467822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1163/2345] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:40.479075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a5914-2bc6-4018-b638-fbb37d2ea9f3 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:40.480907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-939a5914-2bc6-4018-b638-fbb37d2ea9f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:40.538600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a5914-2bc6-4018-b638-fbb37d2ea9f3 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:40.542230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0246f-08ba-4d94-bd3f-bba322e3fb06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:40.542831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0246f-08ba-4d94-bd3f-bba322e3fb06 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:40.543108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0246f-08ba-4d94-bd3f-bba322e3fb06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:40.543436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0246f-08ba-4d94-bd3f-bba322e3fb06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:40.552609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0246f-08ba-4d94-bd3f-bba322e3fb06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:40.554049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0246f-08ba-4d94-bd3f-bba322e3fb06 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:40.564375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1183/2346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:40.651421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753b912-b28c-4c68-90ea-ad154a73ec69 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:40.661334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753b912-b28c-4c68-90ea-ad154a73ec69 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:40.664868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b4ece-8df0-44c6-aa4b-1e1f258c7bb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:40.665510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b4ece-8df0-44c6-aa4b-1e1f258c7bb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:40.665791 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b4ece-8df0-44c6-aa4b-1e1f258c7bb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:40.666068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b4ece-8df0-44c6-aa4b-1e1f258c7bb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:40.724547 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b4ece-8df0-44c6-aa4b-1e1f258c7bb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:40.725921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b4ece-8df0-44c6-aa4b-1e1f258c7bb5 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:40.734729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1164/2347] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:45.469577 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0246f-08ba-4d94-bd3f-bba322e3fb06 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:45.496236 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:45.536885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0246f-08ba-4d94-bd3f-bba322e3fb06 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:45.541485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:45.542123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:45.542405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:45.542682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] PATH_INFO: `/v3/users/e67b70ac47814299a0f50a2c2b27053d` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:45.544339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e67b70ac47814299a0f50a2c2b27053d)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:45.546530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:45.593223 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e67b70ac47814299a0f50a2c2b27053d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:45.606979 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e67b70ac47814299a0f50a2c2b27053d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:45.607779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e67b70ac47814299a0f50a2c2b27053d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:45.614787 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e67b70ac47814299a0f50a2c2b27053d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:45.615424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e67b70ac47814299a0f50a2c2b27053d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:45.628443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1184/2348] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:45:45 2022] DELETE /identity/v3/users/e67b70ac47814299a0f50a2c2b27053d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:45.632975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9213c7-3bd4-4d63-bcec-0676081ab7af None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:45.633520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9213c7-3bd4-4d63-bcec-0676081ab7af None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:45.633693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9213c7-3bd4-4d63-bcec-0676081ab7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:45.653904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-8d9213c7-3bd4-4d63-bcec-0676081ab7af None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:45.655122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8d9213c7-3bd4-4d63-bcec-0676081ab7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:45.714285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1165/2349] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:45.724246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:45.725755 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:45.783529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1909a0-4010-416b-a07d-21a3a30381a8 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:45.787252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e98c11-a84a-4e63-a62b-7c2dad1fe63b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:45.787878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e98c11-a84a-4e63-a62b-7c2dad1fe63b service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:45.788132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e98c11-a84a-4e63-a62b-7c2dad1fe63b service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:45.788405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e98c11-a84a-4e63-a62b-7c2dad1fe63b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:45.797700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e98c11-a84a-4e63-a62b-7c2dad1fe63b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:45.799115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e98c11-a84a-4e63-a62b-7c2dad1fe63b service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:45.808984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1185/2350] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:46.079151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9213c7-3bd4-4d63-bcec-0676081ab7af None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:46.080620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8d9213c7-3bd4-4d63-bcec-0676081ab7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:46.094423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9213c7-3bd4-4d63-bcec-0676081ab7af None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:46.099131 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:46.100417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:46.100909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:46.101224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] PATH_INFO: `/v3/projects/74b33b34704e47baa300f1f8d68843ed` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:46.102368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74b33b34704e47baa300f1f8d68843ed)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:46.104293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:46.148367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:46.156141 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74b33b34704e47baa300f1f8d68843ed', 'request_id': 'req-21f85115-9d3f-4322-a205-4ed9e37a4994'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.162519 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74b33b34704e47baa300f1f8d68843ed', 'request_id': 'req-21f85115-9d3f-4322-a205-4ed9e37a4994'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.171212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74b33b34704e47baa300f1f8d68843ed', 'request_id': 'req-21f85115-9d3f-4322-a205-4ed9e37a4994'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.182556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] The token cache is being invalidate because project 74b33b34704e47baa300f1f8d68843ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:45:46.182848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.183484 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.185942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1166/2351] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:45:46 2022] DELETE /identity/v3/projects/74b33b34704e47baa300f1f8d68843ed => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:45:46.189733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e98c11-a84a-4e63-a62b-7c2dad1fe63b service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:46.200096 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e98c11-a84a-4e63-a62b-7c2dad1fe63b service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:46.244450 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:46.264643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae80c67-0a81-4b49-8c6a-6b37991b61bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:46.265313 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae80c67-0a81-4b49-8c6a-6b37991b61bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:46.265596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae80c67-0a81-4b49-8c6a-6b37991b61bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:46.265873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae80c67-0a81-4b49-8c6a-6b37991b61bf admin admin] PATH_INFO: `/v3/users/e91060c07dd14c0abfdd4c3fd0668e2b` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:46.267001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae80c67-0a81-4b49-8c6a-6b37991b61bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e91060c07dd14c0abfdd4c3fd0668e2b)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:46.269012 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae80c67-0a81-4b49-8c6a-6b37991b61bf admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:46.321556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-fae80c67-0a81-4b49-8c6a-6b37991b61bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e91060c07dd14c0abfdd4c3fd0668e2b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.334399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-fae80c67-0a81-4b49-8c6a-6b37991b61bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e91060c07dd14c0abfdd4c3fd0668e2b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.335149 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-fae80c67-0a81-4b49-8c6a-6b37991b61bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e91060c07dd14c0abfdd4c3fd0668e2b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.342376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-fae80c67-0a81-4b49-8c6a-6b37991b61bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e91060c07dd14c0abfdd4c3fd0668e2b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.343014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-fae80c67-0a81-4b49-8c6a-6b37991b61bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e91060c07dd14c0abfdd4c3fd0668e2b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.357112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1186/2352] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:45:46 2022] DELETE /identity/v3/users/e91060c07dd14c0abfdd4c3fd0668e2b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:46.627883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:46.629374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:46.645446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f85115-9d3f-4322-a205-4ed9e37a4994 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:46.706798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0ba3d-cf03-4fa4-af28-28228c7340e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:46.707443 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0ba3d-cf03-4fa4-af28-28228c7340e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:46.707727 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0ba3d-cf03-4fa4-af28-28228c7340e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:46.708003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0ba3d-cf03-4fa4-af28-28228c7340e0 admin admin] PATH_INFO: `/v3/projects/07297b75cb704e16840cf1f66e329f82` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:46.709348 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0ba3d-cf03-4fa4-af28-28228c7340e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07297b75cb704e16840cf1f66e329f82)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:46.711279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0ba3d-cf03-4fa4-af28-28228c7340e0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:46.758677 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-90b0ba3d-cf03-4fa4-af28-28228c7340e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07297b75cb704e16840cf1f66e329f82', 'request_id': 'req-90b0ba3d-cf03-4fa4-af28-28228c7340e0'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.764877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-90b0ba3d-cf03-4fa4-af28-28228c7340e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07297b75cb704e16840cf1f66e329f82', 'request_id': 'req-90b0ba3d-cf03-4fa4-af28-28228c7340e0'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.772018 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-90b0ba3d-cf03-4fa4-af28-28228c7340e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07297b75cb704e16840cf1f66e329f82', 'request_id': 'req-90b0ba3d-cf03-4fa4-af28-28228c7340e0'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.783850 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-90b0ba3d-cf03-4fa4-af28-28228c7340e0 admin admin] The token cache is being invalidate because project 07297b75cb704e16840cf1f66e329f82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:45:46.784143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-90b0ba3d-cf03-4fa4-af28-28228c7340e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.784783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-90b0ba3d-cf03-4fa4-af28-28228c7340e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:46.786364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1167/2353] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:45:46 2022] DELETE /identity/v3/projects/07297b75cb704e16840cf1f66e329f82 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:46.791278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7c4ca-46ea-4c59-a63e-543c445314a6 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:46.791739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7c4ca-46ea-4c59-a63e-543c445314a6 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:46.791856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7c4ca-46ea-4c59-a63e-543c445314a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:46.812063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e4c7c4ca-46ea-4c59-a63e-543c445314a6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:46.813310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e4c7c4ca-46ea-4c59-a63e-543c445314a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:46.872794 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1187/2354] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:46.880551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e465d52-d254-4bd5-b9c7-3138e1ae6895 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:46.880955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e465d52-d254-4bd5-b9c7-3138e1ae6895 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:46.881267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e465d52-d254-4bd5-b9c7-3138e1ae6895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:46.901136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-7e465d52-d254-4bd5-b9c7-3138e1ae6895 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:46.902341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7e465d52-d254-4bd5-b9c7-3138e1ae6895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:46.910753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1168/2355] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:46.914910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7c4ca-46ea-4c59-a63e-543c445314a6 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:46.925452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7c4ca-46ea-4c59-a63e-543c445314a6 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:46.930127 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec424c-bd2c-4273-957c-fcfee15c5e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:46.930724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec424c-bd2c-4273-957c-fcfee15c5e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:46.931000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec424c-bd2c-4273-957c-fcfee15c5e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:46.931270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec424c-bd2c-4273-957c-fcfee15c5e7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:46.931911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec424c-bd2c-4273-957c-fcfee15c5e7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:45:46.932211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec424c-bd2c-4273-957c-fcfee15c5e7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:46.933820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec424c-bd2c-4273-957c-fcfee15c5e7b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:46.946925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1188/2356] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:45:46.951533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e465d52-d254-4bd5-b9c7-3138e1ae6895 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:46.961545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e465d52-d254-4bd5-b9c7-3138e1ae6895 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:46.966933 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc232b-5d31-4208-ace8-ea5bc0740ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:46.967673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc232b-5d31-4208-ace8-ea5bc0740ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:46.968025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc232b-5d31-4208-ace8-ea5bc0740ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:46.968368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc232b-5d31-4208-ace8-ea5bc0740ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:46.969080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc232b-5d31-4208-ace8-ea5bc0740ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:46.971079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc232b-5d31-4208-ace8-ea5bc0740ac9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:47.002001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1169/2357] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:46 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:45:47.006612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec424c-bd2c-4273-957c-fcfee15c5e7b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:47.018083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec424c-bd2c-4273-957c-fcfee15c5e7b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:47.070352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:47.084893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894befd-55f8-4787-a66b-fc8311113b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:47.085618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894befd-55f8-4787-a66b-fc8311113b1e admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.085968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894befd-55f8-4787-a66b-fc8311113b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.086266 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894befd-55f8-4787-a66b-fc8311113b1e admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:47.087097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894befd-55f8-4787-a66b-fc8311113b1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:47.091242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894befd-55f8-4787-a66b-fc8311113b1e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:47.114595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1189/2358] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:45:47 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:45:47.118890 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc232b-5d31-4208-ace8-ea5bc0740ac9 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:47.129167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc232b-5d31-4208-ace8-ea5bc0740ac9 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:47.133822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcc30b-64fc-42fc-8392-f7c89ea3358c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:47.134418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcc30b-64fc-42fc-8392-f7c89ea3358c admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.134696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcc30b-64fc-42fc-8392-f7c89ea3358c admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.134966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcc30b-64fc-42fc-8392-f7c89ea3358c admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:47.135606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcc30b-64fc-42fc-8392-f7c89ea3358c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:47.137758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcc30b-64fc-42fc-8392-f7c89ea3358c admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:47.148573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1170/2359] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:45:47.152775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894befd-55f8-4787-a66b-fc8311113b1e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:47.163635 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894befd-55f8-4787-a66b-fc8311113b1e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:47.168564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a2857-a387-4a3e-90b6-cb6c6bb1b477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:47.169201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a2857-a387-4a3e-90b6-cb6c6bb1b477 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.169485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a2857-a387-4a3e-90b6-cb6c6bb1b477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.169756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a2857-a387-4a3e-90b6-cb6c6bb1b477 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:47.170399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a2857-a387-4a3e-90b6-cb6c6bb1b477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:47.172261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a2857-a387-4a3e-90b6-cb6c6bb1b477 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:47.185798 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1190/2360] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:47.190257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcc30b-64fc-42fc-8392-f7c89ea3358c admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:47.203210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcc30b-64fc-42fc-8392-f7c89ea3358c admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:47.209043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e284f-6251-434c-bba4-5649e2ba4959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:47.209724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e284f-6251-434c-bba4-5649e2ba4959 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.210014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e284f-6251-434c-bba4-5649e2ba4959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.210316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e284f-6251-434c-bba4-5649e2ba4959 admin admin] PATH_INFO: `/v3/projects/5c481f1c49c04c53a8479c2a46d4eb6c/users/9ab312f7044a48dbbbf1709855b1a6af/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:47.230109 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e284f-6251-434c-bba4-5649e2ba4959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c481f1c49c04c53a8479c2a46d4eb6c, user_id=9ab312f7044a48dbbbf1709855b1a6af, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:47.233847 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e284f-6251-434c-bba4-5649e2ba4959 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:47.236869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:47.252599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1171/2361] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:47 2022] PUT /identity/v3/projects/5c481f1c49c04c53a8479c2a46d4eb6c/users/9ab312f7044a48dbbbf1709855b1a6af/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:47.266220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cf529-2135-479f-9ea8-b0ea05690e15 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.266517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cf529-2135-479f-9ea8-b0ea05690e15 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.266768 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cf529-2135-479f-9ea8-b0ea05690e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:47.299647 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-382cf529-2135-479f-9ea8-b0ea05690e15 None None] MFA Rules not processed for user `9ab312f7044a48dbbbf1709855b1a6af`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:47.300965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-382cf529-2135-479f-9ea8-b0ea05690e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:47.375123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1191/2362] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:47.379584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e284f-6251-434c-bba4-5649e2ba4959 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:47.443604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e284f-6251-434c-bba4-5649e2ba4959 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:47.448223 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8001ba-b182-43bf-b17a-d374f6205794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:47.448825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8001ba-b182-43bf-b17a-d374f6205794 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.449117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8001ba-b182-43bf-b17a-d374f6205794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.449403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8001ba-b182-43bf-b17a-d374f6205794 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:47.450073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8001ba-b182-43bf-b17a-d374f6205794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:47.451969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8001ba-b182-43bf-b17a-d374f6205794 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:47.479775 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1172/2363] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:47 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:47.484602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cf529-2135-479f-9ea8-b0ea05690e15 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:47.495824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cf529-2135-479f-9ea8-b0ea05690e15 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:47.553723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8ffc7-564a-489a-aa8d-644a84d8ea3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:47.554330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8ffc7-564a-489a-aa8d-644a84d8ea3f admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.554612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8ffc7-564a-489a-aa8d-644a84d8ea3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.554950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8ffc7-564a-489a-aa8d-644a84d8ea3f admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:47.555511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8ffc7-564a-489a-aa8d-644a84d8ea3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:47.557514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8ffc7-564a-489a-aa8d-644a84d8ea3f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:47.579359 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1192/2364] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:45:47 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:47.583846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8001ba-b182-43bf-b17a-d374f6205794 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:47.594960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8001ba-b182-43bf-b17a-d374f6205794 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:47.599825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8fa89-9ed4-4840-947a-4100a7c2216a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:47.600431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8fa89-9ed4-4840-947a-4100a7c2216a admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.600713 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8fa89-9ed4-4840-947a-4100a7c2216a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.600986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8fa89-9ed4-4840-947a-4100a7c2216a admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:47.601661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8fa89-9ed4-4840-947a-4100a7c2216a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:47.603278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8fa89-9ed4-4840-947a-4100a7c2216a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:47.615151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1173/2365] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:47.619327 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8ffc7-564a-489a-aa8d-644a84d8ea3f admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:47.629508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8ffc7-564a-489a-aa8d-644a84d8ea3f admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:47.634153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61c139-b70f-4fea-b28e-388766f3467a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:47.634794 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61c139-b70f-4fea-b28e-388766f3467a admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.635095 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61c139-b70f-4fea-b28e-388766f3467a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.635351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61c139-b70f-4fea-b28e-388766f3467a admin admin] PATH_INFO: `/v3/projects/f514110850a142b7a3d3be2a06e67cf6/users/dae3366cf69d4415b5178623fe53a76a/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:47.659216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61c139-b70f-4fea-b28e-388766f3467a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f514110850a142b7a3d3be2a06e67cf6, user_id=dae3366cf69d4415b5178623fe53a76a, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:47.661941 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61c139-b70f-4fea-b28e-388766f3467a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:47.676253 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1193/2366] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:47 2022] PUT /identity/v3/projects/f514110850a142b7a3d3be2a06e67cf6/users/dae3366cf69d4415b5178623fe53a76a/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:45:47.680210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8fa89-9ed4-4840-947a-4100a7c2216a admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:47.690498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8fa89-9ed4-4840-947a-4100a7c2216a admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:47.747940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117539c8-75bc-4404-bc2e-961c23d0407b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:47.748495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117539c8-75bc-4404-bc2e-961c23d0407b admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.748774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117539c8-75bc-4404-bc2e-961c23d0407b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.749045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117539c8-75bc-4404-bc2e-961c23d0407b admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:47.749714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117539c8-75bc-4404-bc2e-961c23d0407b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:47.751329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117539c8-75bc-4404-bc2e-961c23d0407b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:47.761149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1174/2367] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:47.765813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61c139-b70f-4fea-b28e-388766f3467a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:47.775877 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61c139-b70f-4fea-b28e-388766f3467a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:47.781233 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ad450-3722-4560-b3ee-0c5831c072d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:47.781874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ad450-3722-4560-b3ee-0c5831c072d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.782150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ad450-3722-4560-b3ee-0c5831c072d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.782422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ad450-3722-4560-b3ee-0c5831c072d7 admin admin] PATH_INFO: `/v3/projects/f514110850a142b7a3d3be2a06e67cf6/users/dae3366cf69d4415b5178623fe53a76a/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:47.784286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ad450-3722-4560-b3ee-0c5831c072d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f514110850a142b7a3d3be2a06e67cf6, user_id=dae3366cf69d4415b5178623fe53a76a, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:47.786919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ad450-3722-4560-b3ee-0c5831c072d7 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:47.797337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1194/2368] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:47 2022] PUT /identity/v3/projects/f514110850a142b7a3d3be2a06e67cf6/users/dae3366cf69d4415b5178623fe53a76a/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:47.809522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e6ce4-f602-40d2-83bf-26b699b61240 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.809808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e6ce4-f602-40d2-83bf-26b699b61240 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.810082 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e6ce4-f602-40d2-83bf-26b699b61240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:47.838131 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-6b6e6ce4-f602-40d2-83bf-26b699b61240 None None] MFA Rules not processed for user `dae3366cf69d4415b5178623fe53a76a`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:47.839340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6b6e6ce4-f602-40d2-83bf-26b699b61240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:47.889644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:47.908857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1175/2369] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:47.920242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ad450-3722-4560-b3ee-0c5831c072d7 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:47.922191 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-458ad450-3722-4560-b3ee-0c5831c072d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:47.936805 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:47.986540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ad450-3722-4560-b3ee-0c5831c072d7 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:47.990391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d493c-97bd-41b9-a10b-33fbc3a50ea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:47.990998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d493c-97bd-41b9-a10b-33fbc3a50ea8 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:47.991279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d493c-97bd-41b9-a10b-33fbc3a50ea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:47.991553 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d493c-97bd-41b9-a10b-33fbc3a50ea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:48.001575 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d493c-97bd-41b9-a10b-33fbc3a50ea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:48.002929 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d493c-97bd-41b9-a10b-33fbc3a50ea8 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:48.013030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1195/2370] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:48.086101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e6ce4-f602-40d2-83bf-26b699b61240 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:48.095696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e6ce4-f602-40d2-83bf-26b699b61240 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:48.099536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a78f3-dc3e-46f3-94f3-5cd70a99c41b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:48.100130 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a78f3-dc3e-46f3-94f3-5cd70a99c41b service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:48.100410 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a78f3-dc3e-46f3-94f3-5cd70a99c41b service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:48.100684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a78f3-dc3e-46f3-94f3-5cd70a99c41b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:48.157708 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a78f3-dc3e-46f3-94f3-5cd70a99c41b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:48.159144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a78f3-dc3e-46f3-94f3-5cd70a99c41b service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:48.168060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1176/2371] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:48.447135 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d493c-97bd-41b9-a10b-33fbc3a50ea8 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:48.513677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d493c-97bd-41b9-a10b-33fbc3a50ea8 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:48.518659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:48.519307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:48.519588 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:48.519864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] PATH_INFO: `/v3/users/9ab312f7044a48dbbbf1709855b1a6af` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:48.521160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab312f7044a48dbbbf1709855b1a6af)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:48.523040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:48.575242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab312f7044a48dbbbf1709855b1a6af'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:48.587909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab312f7044a48dbbbf1709855b1a6af'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:48.588643 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab312f7044a48dbbbf1709855b1a6af'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:48.599105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab312f7044a48dbbbf1709855b1a6af'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:48.599868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab312f7044a48dbbbf1709855b1a6af'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:48.614792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1196/2372] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:45:48 2022] DELETE /identity/v3/users/9ab312f7044a48dbbbf1709855b1a6af => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:48.619349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49049ca-ebfd-4170-9bc3-8f5cd996c6db None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:48.619640 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49049ca-ebfd-4170-9bc3-8f5cd996c6db None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:48.619923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49049ca-ebfd-4170-9bc3-8f5cd996c6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:48.639363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-f49049ca-ebfd-4170-9bc3-8f5cd996c6db None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:48.640571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f49049ca-ebfd-4170-9bc3-8f5cd996c6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:48.709304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1177/2373] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:48.719228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:48.720645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:48.756478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:48.781216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e6b88-6a5b-4f4c-9e70-d7cb47847ebe admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:48.784797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f805436-e2fd-4a03-8ff0-536b5704d2f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:48.785441 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f805436-e2fd-4a03-8ff0-536b5704d2f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:48.785722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f805436-e2fd-4a03-8ff0-536b5704d2f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:48.785995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f805436-e2fd-4a03-8ff0-536b5704d2f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:48.795175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f805436-e2fd-4a03-8ff0-536b5704d2f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:48.796872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f805436-e2fd-4a03-8ff0-536b5704d2f0 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:48.806410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1197/2374] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:49.084273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49049ca-ebfd-4170-9bc3-8f5cd996c6db None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:49.086994 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f49049ca-ebfd-4170-9bc3-8f5cd996c6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:49.101819 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49049ca-ebfd-4170-9bc3-8f5cd996c6db None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:49.108093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:49.117481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:49.117774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:49.118060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] PATH_INFO: `/v3/projects/5c481f1c49c04c53a8479c2a46d4eb6c` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:49.119439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c481f1c49c04c53a8479c2a46d4eb6c)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:49.121438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:49.173000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c481f1c49c04c53a8479c2a46d4eb6c', 'request_id': 'req-f1d1ac39-db7f-4db4-a941-329f66f37bb2'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.178585 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c481f1c49c04c53a8479c2a46d4eb6c', 'request_id': 'req-f1d1ac39-db7f-4db4-a941-329f66f37bb2'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.188966 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c481f1c49c04c53a8479c2a46d4eb6c', 'request_id': 'req-f1d1ac39-db7f-4db4-a941-329f66f37bb2'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.201267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] The token cache is being invalidate because project 5c481f1c49c04c53a8479c2a46d4eb6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:45:49.201563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.202210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.204002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1178/2375] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:45:49 2022] DELETE /identity/v3/projects/5c481f1c49c04c53a8479c2a46d4eb6c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:49.208091 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f805436-e2fd-4a03-8ff0-536b5704d2f0 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:49.218354 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f805436-e2fd-4a03-8ff0-536b5704d2f0 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:49.278648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fc9b8-4e74-40be-a65b-ae8b180138bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:49.278648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fc9b8-4e74-40be-a65b-ae8b180138bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:49.280089 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fc9b8-4e74-40be-a65b-ae8b180138bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:49.280089 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fc9b8-4e74-40be-a65b-ae8b180138bf admin admin] PATH_INFO: `/v3/users/dae3366cf69d4415b5178623fe53a76a` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:49.280486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fc9b8-4e74-40be-a65b-ae8b180138bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae3366cf69d4415b5178623fe53a76a)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:49.282708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fc9b8-4e74-40be-a65b-ae8b180138bf admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:49.334911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-183fc9b8-4e74-40be-a65b-ae8b180138bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae3366cf69d4415b5178623fe53a76a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.349190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-183fc9b8-4e74-40be-a65b-ae8b180138bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae3366cf69d4415b5178623fe53a76a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.349937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-183fc9b8-4e74-40be-a65b-ae8b180138bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae3366cf69d4415b5178623fe53a76a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.359698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-183fc9b8-4e74-40be-a65b-ae8b180138bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae3366cf69d4415b5178623fe53a76a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.360437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-183fc9b8-4e74-40be-a65b-ae8b180138bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae3366cf69d4415b5178623fe53a76a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.382997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1198/2376] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:45:49 2022] DELETE /identity/v3/users/dae3366cf69d4415b5178623fe53a76a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:49.677065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:49.681548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:49.705385 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1ac39-db7f-4db4-a941-329f66f37bb2 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:49.723091 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:49.798255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:49.798910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:49.799174 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:49.799448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30 admin admin] PATH_INFO: `/v3/projects/f514110850a142b7a3d3be2a06e67cf6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:49.800610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f514110850a142b7a3d3be2a06e67cf6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:49.802579 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:49.854458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f514110850a142b7a3d3be2a06e67cf6', 'request_id': 'req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.863334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f514110850a142b7a3d3be2a06e67cf6', 'request_id': 'req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.873838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f514110850a142b7a3d3be2a06e67cf6', 'request_id': 'req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.886467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30 admin admin] The token cache is being invalidate because project f514110850a142b7a3d3be2a06e67cf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:45:49.886811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.887549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-17c18dba-4e62-4e2e-a53a-71bc6ba74e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:49.892246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1179/2377] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:45:49 2022] DELETE /identity/v3/projects/f514110850a142b7a3d3be2a06e67cf6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:49.894124 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05036df4-61a0-4de5-b920-8dd88f7d2bc1 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:49.894592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05036df4-61a0-4de5-b920-8dd88f7d2bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:49.894700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05036df4-61a0-4de5-b920-8dd88f7d2bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:49.930823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-05036df4-61a0-4de5-b920-8dd88f7d2bc1 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:49.931856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-05036df4-61a0-4de5-b920-8dd88f7d2bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:50.022328 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:50.043814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1199/2378] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:50.052311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fea8b-5d90-463b-9b87-869d76bbc7b9 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:50.052801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fea8b-5d90-463b-9b87-869d76bbc7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:50.053276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fea8b-5d90-463b-9b87-869d76bbc7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:50.077788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-8e3fea8b-5d90-463b-9b87-869d76bbc7b9 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:50.079011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8e3fea8b-5d90-463b-9b87-869d76bbc7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:50.087954 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1180/2379] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:50.092503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05036df4-61a0-4de5-b920-8dd88f7d2bc1 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:50.105545 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05036df4-61a0-4de5-b920-8dd88f7d2bc1 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:50.113511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc17f8-7625-440f-b4a5-336810292ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:50.114624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc17f8-7625-440f-b4a5-336810292ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:50.114911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc17f8-7625-440f-b4a5-336810292ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:50.115189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc17f8-7625-440f-b4a5-336810292ce6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:50.115852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc17f8-7625-440f-b4a5-336810292ce6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:45:50.116157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc17f8-7625-440f-b4a5-336810292ce6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:50.117861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc17f8-7625-440f-b4a5-336810292ce6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:50.133139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1200/2380] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:45:50.136643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fea8b-5d90-463b-9b87-869d76bbc7b9 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:50.147106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fea8b-5d90-463b-9b87-869d76bbc7b9 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:50.152485 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812c4db-b3cd-48a8-9508-de2aef1838be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:50.152923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812c4db-b3cd-48a8-9508-de2aef1838be admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:50.154921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812c4db-b3cd-48a8-9508-de2aef1838be admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:50.155428 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812c4db-b3cd-48a8-9508-de2aef1838be admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:50.156161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812c4db-b3cd-48a8-9508-de2aef1838be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:50.158077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812c4db-b3cd-48a8-9508-de2aef1838be admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:50.193127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1181/2381] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:50 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:50.195842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc17f8-7625-440f-b4a5-336810292ce6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:50.207084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc17f8-7625-440f-b4a5-336810292ce6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:50.272580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08531f-97b8-4793-8739-ceaa3566df2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:50.273372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08531f-97b8-4793-8739-ceaa3566df2c admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:50.273719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08531f-97b8-4793-8739-ceaa3566df2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:50.274057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08531f-97b8-4793-8739-ceaa3566df2c admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:50.274698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08531f-97b8-4793-8739-ceaa3566df2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:50.276870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08531f-97b8-4793-8739-ceaa3566df2c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:50.301175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1201/2382] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:45:50 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:50.305301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812c4db-b3cd-48a8-9508-de2aef1838be admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:50.316302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812c4db-b3cd-48a8-9508-de2aef1838be admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:50.321004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948be063-6c24-43f7-a4c3-8468d4169f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:50.321684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948be063-6c24-43f7-a4c3-8468d4169f07 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:50.321965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948be063-6c24-43f7-a4c3-8468d4169f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:50.322238 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948be063-6c24-43f7-a4c3-8468d4169f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:50.322901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948be063-6c24-43f7-a4c3-8468d4169f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:50.324497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948be063-6c24-43f7-a4c3-8468d4169f07 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:50.335238 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1182/2383] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:50.339299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08531f-97b8-4793-8739-ceaa3566df2c admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:50.349739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08531f-97b8-4793-8739-ceaa3566df2c admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:50.354521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613ed3f-70f9-4680-babc-9c4938715563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:50.355126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613ed3f-70f9-4680-babc-9c4938715563 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:50.355407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613ed3f-70f9-4680-babc-9c4938715563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:50.355687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613ed3f-70f9-4680-babc-9c4938715563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:50.356389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613ed3f-70f9-4680-babc-9c4938715563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:50.358025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613ed3f-70f9-4680-babc-9c4938715563 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:50.372788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1202/2384] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:50.374040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948be063-6c24-43f7-a4c3-8468d4169f07 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:50.385134 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948be063-6c24-43f7-a4c3-8468d4169f07 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:50.391561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1497d98-308c-4efe-865f-719e9afceb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:50.392225 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1497d98-308c-4efe-865f-719e9afceb1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:50.392662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1497d98-308c-4efe-865f-719e9afceb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:50.392940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1497d98-308c-4efe-865f-719e9afceb1a admin admin] PATH_INFO: `/v3/projects/15e8870bfc314c27a94b26e2acf9514c/users/ec213f0f002240e8932670cc47ed9632/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:50.412500 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1497d98-308c-4efe-865f-719e9afceb1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e8870bfc314c27a94b26e2acf9514c, user_id=ec213f0f002240e8932670cc47ed9632, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:50.415435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1497d98-308c-4efe-865f-719e9afceb1a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:50.432340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1183/2385] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:50 2022] PUT /identity/v3/projects/15e8870bfc314c27a94b26e2acf9514c/users/ec213f0f002240e8932670cc47ed9632/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:50.444881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bc316-ecce-462e-83f6-e36750bfb56f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:50.445068 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bc316-ecce-462e-83f6-e36750bfb56f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:50.445441 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bc316-ecce-462e-83f6-e36750bfb56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:50.477977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-1a0bc316-ecce-462e-83f6-e36750bfb56f None None] MFA Rules not processed for user `ec213f0f002240e8932670cc47ed9632`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:50.479323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-1a0bc316-ecce-462e-83f6-e36750bfb56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:50.547546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1203/2386] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:50.559311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1497d98-308c-4efe-865f-719e9afceb1a admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:50.562161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e1497d98-308c-4efe-865f-719e9afceb1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:50.614592 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:50.635108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1497d98-308c-4efe-865f-719e9afceb1a admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:50.641198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458d309-1c5c-4609-bf7e-6a95ff75f973 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:50.642129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458d309-1c5c-4609-bf7e-6a95ff75f973 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:50.642255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458d309-1c5c-4609-bf7e-6a95ff75f973 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:50.642662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458d309-1c5c-4609-bf7e-6a95ff75f973 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:50.654891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458d309-1c5c-4609-bf7e-6a95ff75f973 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:50.656425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458d309-1c5c-4609-bf7e-6a95ff75f973 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:50.667874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1184/2387] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:50.690760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bc316-ecce-462e-83f6-e36750bfb56f None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:50.750146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:50.766691 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bc316-ecce-462e-83f6-e36750bfb56f None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:50.773925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:50.774515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:50.774850 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:50.775172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] PATH_INFO: `/v3/users/ec213f0f002240e8932670cc47ed9632` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:50.776457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec213f0f002240e8932670cc47ed9632)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:50.781153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:50.850180 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec213f0f002240e8932670cc47ed9632'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:50.865719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec213f0f002240e8932670cc47ed9632'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:50.866404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec213f0f002240e8932670cc47ed9632'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:50.874206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec213f0f002240e8932670cc47ed9632'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:50.874917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec213f0f002240e8932670cc47ed9632'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:50.891012 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1204/2388] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:45:50 2022] DELETE /identity/v3/users/ec213f0f002240e8932670cc47ed9632 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:50.895435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef71d2-9685-481a-83df-69cc62864139 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:50.895934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef71d2-9685-481a-83df-69cc62864139 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:50.895934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef71d2-9685-481a-83df-69cc62864139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:50.916562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-23ef71d2-9685-481a-83df-69cc62864139 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:50.917792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-23ef71d2-9685-481a-83df-69cc62864139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:50.985183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1185/2389] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:50.995414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:50.997568 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:51.102894 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86317d80-8373-476b-9c5c-98de007f7928 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:51.106587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60f6a4-ffb0-4a9d-9f50-7155a6791845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:51.107202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60f6a4-ffb0-4a9d-9f50-7155a6791845 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:51.107483 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60f6a4-ffb0-4a9d-9f50-7155a6791845 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:51.107761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60f6a4-ffb0-4a9d-9f50-7155a6791845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:51.117148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60f6a4-ffb0-4a9d-9f50-7155a6791845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:51.118521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60f6a4-ffb0-4a9d-9f50-7155a6791845 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:51.128757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1205/2390] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:51.561417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef71d2-9685-481a-83df-69cc62864139 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:51.563545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-23ef71d2-9685-481a-83df-69cc62864139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:51.578787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef71d2-9685-481a-83df-69cc62864139 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:51.584915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:51.585933 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:51.586371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:51.586831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979 admin admin] PATH_INFO: `/v3/projects/15e8870bfc314c27a94b26e2acf9514c` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:51.588379 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e8870bfc314c27a94b26e2acf9514c)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:51.591121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:51.649269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e8870bfc314c27a94b26e2acf9514c', 'request_id': 'req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:51.655666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e8870bfc314c27a94b26e2acf9514c', 'request_id': 'req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:51.664265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e8870bfc314c27a94b26e2acf9514c', 'request_id': 'req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:51.677480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979 admin admin] The token cache is being invalidate because project 15e8870bfc314c27a94b26e2acf9514c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:45:51.677750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:51.678788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-dcc305ab-5c46-4f25-9b0d-8400b5a8d979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:51.680558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1186/2391] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:45:51 2022] DELETE /identity/v3/projects/15e8870bfc314c27a94b26e2acf9514c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:51.685896 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df957957-583d-4cf7-865b-730df88c6bb1 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:51.686319 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df957957-583d-4cf7-865b-730df88c6bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:51.686605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df957957-583d-4cf7-865b-730df88c6bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:51.706834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-df957957-583d-4cf7-865b-730df88c6bb1 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:51.708053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-df957957-583d-4cf7-865b-730df88c6bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:51.771051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1206/2392] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:51.838560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b132cf-3b28-422d-8216-6cc98bf5d034 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:51.838848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b132cf-3b28-422d-8216-6cc98bf5d034 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:51.839671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b132cf-3b28-422d-8216-6cc98bf5d034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:51.865647 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-84b132cf-3b28-422d-8216-6cc98bf5d034 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:51.866626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-84b132cf-3b28-422d-8216-6cc98bf5d034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:51.875341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1187/2393] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:51.879971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df957957-583d-4cf7-865b-730df88c6bb1 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:51.891027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df957957-583d-4cf7-865b-730df88c6bb1 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:51.895919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea79717-786e-441d-98f7-bdb49ba57e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:51.896600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea79717-786e-441d-98f7-bdb49ba57e93 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:51.896954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea79717-786e-441d-98f7-bdb49ba57e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:51.897351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea79717-786e-441d-98f7-bdb49ba57e93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:51.898078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea79717-786e-441d-98f7-bdb49ba57e93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:45:51.898452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea79717-786e-441d-98f7-bdb49ba57e93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:51.900116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea79717-786e-441d-98f7-bdb49ba57e93 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:51.909053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:51.913149 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1207/2394] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:45:51.917681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b132cf-3b28-422d-8216-6cc98bf5d034 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:51.927357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b132cf-3b28-422d-8216-6cc98bf5d034 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:51.932286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af9868-c748-4b6d-9ed7-e586365512b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:51.933185 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af9868-c748-4b6d-9ed7-e586365512b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:51.933468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af9868-c748-4b6d-9ed7-e586365512b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:51.933734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af9868-c748-4b6d-9ed7-e586365512b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:51.934884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af9868-c748-4b6d-9ed7-e586365512b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:51.936810 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af9868-c748-4b6d-9ed7-e586365512b6 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:51.970562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1188/2395] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:51 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:51.972262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea79717-786e-441d-98f7-bdb49ba57e93 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:51.983497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea79717-786e-441d-98f7-bdb49ba57e93 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:52.047402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899100fc-ed46-49d8-8932-500114706f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:52.048060 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899100fc-ed46-49d8-8932-500114706f5a admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.048274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899100fc-ed46-49d8-8932-500114706f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.048545 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899100fc-ed46-49d8-8932-500114706f5a admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.049207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899100fc-ed46-49d8-8932-500114706f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:52.051109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899100fc-ed46-49d8-8932-500114706f5a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:52.074027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1208/2396] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:45:51 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:52.078184 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af9868-c748-4b6d-9ed7-e586365512b6 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:52.088721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af9868-c748-4b6d-9ed7-e586365512b6 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:52.094056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5eff0b-570c-4064-a7df-2c766d35268e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:52.094839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5eff0b-570c-4064-a7df-2c766d35268e admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.095177 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5eff0b-570c-4064-a7df-2c766d35268e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.095656 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5eff0b-570c-4064-a7df-2c766d35268e admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.096370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5eff0b-570c-4064-a7df-2c766d35268e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:52.098105 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5eff0b-570c-4064-a7df-2c766d35268e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:52.113604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1189/2397] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:52.117354 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899100fc-ed46-49d8-8932-500114706f5a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:52.128683 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899100fc-ed46-49d8-8932-500114706f5a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:52.134015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264ec55-893a-4057-92a5-c6a9ac74c4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:52.134693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264ec55-893a-4057-92a5-c6a9ac74c4ad admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.134927 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264ec55-893a-4057-92a5-c6a9ac74c4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.135197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264ec55-893a-4057-92a5-c6a9ac74c4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.136410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264ec55-893a-4057-92a5-c6a9ac74c4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:52.138052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264ec55-893a-4057-92a5-c6a9ac74c4ad admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:52.149131 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1209/2398] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:52.153391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5eff0b-570c-4064-a7df-2c766d35268e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:52.163576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5eff0b-570c-4064-a7df-2c766d35268e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:52.172167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4758d-6fa2-4127-b715-f1d68b94bf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:52.172167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4758d-6fa2-4127-b715-f1d68b94bf47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.172167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4758d-6fa2-4127-b715-f1d68b94bf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.172167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4758d-6fa2-4127-b715-f1d68b94bf47 admin admin] PATH_INFO: `/v3/projects/6a047cd61d4c4e3fa55bfa0084cbd167/users/1687d75e941d429a9b5069a2d8136813/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.190654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4758d-6fa2-4127-b715-f1d68b94bf47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a047cd61d4c4e3fa55bfa0084cbd167, user_id=1687d75e941d429a9b5069a2d8136813, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:52.193227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4758d-6fa2-4127-b715-f1d68b94bf47 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:52.207111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1190/2399] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:52 2022] PUT /identity/v3/projects/6a047cd61d4c4e3fa55bfa0084cbd167/users/1687d75e941d429a9b5069a2d8136813/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:45:52.217470 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619e1a2-18a0-4bf7-aa35-bad8d92bba33 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.217763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619e1a2-18a0-4bf7-aa35-bad8d92bba33 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.218035 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619e1a2-18a0-4bf7-aa35-bad8d92bba33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.251116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-9619e1a2-18a0-4bf7-aa35-bad8d92bba33 None None] MFA Rules not processed for user `1687d75e941d429a9b5069a2d8136813`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:52.252486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9619e1a2-18a0-4bf7-aa35-bad8d92bba33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:52.327985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1210/2400] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:52.332886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4758d-6fa2-4127-b715-f1d68b94bf47 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:52.338196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:52.411466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4758d-6fa2-4127-b715-f1d68b94bf47 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:52.416657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dffec1-4eb6-4646-b57f-6abed8acd404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:52.417306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dffec1-4eb6-4646-b57f-6abed8acd404 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.417586 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dffec1-4eb6-4646-b57f-6abed8acd404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.417858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dffec1-4eb6-4646-b57f-6abed8acd404 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.419058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dffec1-4eb6-4646-b57f-6abed8acd404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:52.421278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dffec1-4eb6-4646-b57f-6abed8acd404 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:52.452633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1191/2401] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:52 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:52.454245 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619e1a2-18a0-4bf7-aa35-bad8d92bba33 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:52.465446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619e1a2-18a0-4bf7-aa35-bad8d92bba33 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:52.542336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f383d-84a8-4dd0-a269-5edd5d99767c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:52.543037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f383d-84a8-4dd0-a269-5edd5d99767c admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.543474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f383d-84a8-4dd0-a269-5edd5d99767c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.544317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f383d-84a8-4dd0-a269-5edd5d99767c admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.545050 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f383d-84a8-4dd0-a269-5edd5d99767c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:52.547469 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f383d-84a8-4dd0-a269-5edd5d99767c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:52.570574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:52.579055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1211/2402] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:45:52 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:52.584291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dffec1-4eb6-4646-b57f-6abed8acd404 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:52.596168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dffec1-4eb6-4646-b57f-6abed8acd404 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:52.600892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643249f-010b-4d38-9e50-12690e3a74d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:52.601734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643249f-010b-4d38-9e50-12690e3a74d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.601985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643249f-010b-4d38-9e50-12690e3a74d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.602257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643249f-010b-4d38-9e50-12690e3a74d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.603051 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643249f-010b-4d38-9e50-12690e3a74d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:52.604662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643249f-010b-4d38-9e50-12690e3a74d3 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:52.616390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1192/2403] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:52.620479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f383d-84a8-4dd0-a269-5edd5d99767c admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:52.631496 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f383d-84a8-4dd0-a269-5edd5d99767c admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:52.636688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3813e26-fb7d-4c03-856b-3d89ff86a421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:52.637458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3813e26-fb7d-4c03-856b-3d89ff86a421 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.637780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3813e26-fb7d-4c03-856b-3d89ff86a421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.638109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3813e26-fb7d-4c03-856b-3d89ff86a421 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.638919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3813e26-fb7d-4c03-856b-3d89ff86a421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:52.640528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3813e26-fb7d-4c03-856b-3d89ff86a421 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:52.651028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1212/2404] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:52.655729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643249f-010b-4d38-9e50-12690e3a74d3 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:52.666211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643249f-010b-4d38-9e50-12690e3a74d3 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:52.671778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4ec77-8cfc-44b3-837b-f4d9d4cd93f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:52.672442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4ec77-8cfc-44b3-837b-f4d9d4cd93f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.672717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4ec77-8cfc-44b3-837b-f4d9d4cd93f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.672987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4ec77-8cfc-44b3-837b-f4d9d4cd93f9 admin admin] PATH_INFO: `/v3/projects/4036118fcf434626bc58ddf3800b3b65/users/72962efcc4e44c6abd7d5bab45f8290a/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.689165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4ec77-8cfc-44b3-837b-f4d9d4cd93f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4036118fcf434626bc58ddf3800b3b65, user_id=72962efcc4e44c6abd7d5bab45f8290a, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:52.691858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4ec77-8cfc-44b3-837b-f4d9d4cd93f9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:52.707389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1193/2405] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:52 2022] PUT /identity/v3/projects/4036118fcf434626bc58ddf3800b3b65/users/72962efcc4e44c6abd7d5bab45f8290a/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:52.719667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd88d48-c7d2-49cf-8e83-2856f6728144 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.719953 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd88d48-c7d2-49cf-8e83-2856f6728144 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.720226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd88d48-c7d2-49cf-8e83-2856f6728144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.750851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-cdd88d48-c7d2-49cf-8e83-2856f6728144 None None] MFA Rules not processed for user `72962efcc4e44c6abd7d5bab45f8290a`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:52.752199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-cdd88d48-c7d2-49cf-8e83-2856f6728144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:52.819198 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1213/2406] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:52.823985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4ec77-8cfc-44b3-837b-f4d9d4cd93f9 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:52.888111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4ec77-8cfc-44b3-837b-f4d9d4cd93f9 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:52.893069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e5fde-143c-4b19-bf90-b36dea7c6341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:52.893699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e5fde-143c-4b19-bf90-b36dea7c6341 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.893975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e5fde-143c-4b19-bf90-b36dea7c6341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.894244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e5fde-143c-4b19-bf90-b36dea7c6341 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.894887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e5fde-143c-4b19-bf90-b36dea7c6341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:52.896837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e5fde-143c-4b19-bf90-b36dea7c6341 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:52.927425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1194/2407] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:45:52 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:52.929619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fda5f6-1110-4bd7-bc85-3dab6b471346 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:52.929908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fda5f6-1110-4bd7-bc85-3dab6b471346 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:52.930184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fda5f6-1110-4bd7-bc85-3dab6b471346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:52.931584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e5fde-143c-4b19-bf90-b36dea7c6341 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:52.952440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-82fda5f6-1110-4bd7-bc85-3dab6b471346 None None] MFA Rules not processed for user `fe0fcdeafae645a582afcee3dde79ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:52.953820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-82fda5f6-1110-4bd7-bc85-3dab6b471346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:52.977468 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:53.001299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e5fde-143c-4b19-bf90-b36dea7c6341 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:53.006015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd54a64-57be-4c05-9ab9-e6990d86c93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:53.006613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd54a64-57be-4c05-9ab9-e6990d86c93b admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:53.006889 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd54a64-57be-4c05-9ab9-e6990d86c93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:53.007519 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd54a64-57be-4c05-9ab9-e6990d86c93b admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:53.008155 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd54a64-57be-4c05-9ab9-e6990d86c93b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:53.010106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd54a64-57be-4c05-9ab9-e6990d86c93b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:53.013497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1214/2408] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:45:53.034197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1195/2409] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:45:52 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:45:53.039052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fda5f6-1110-4bd7-bc85-3dab6b471346 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:53.051112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fda5f6-1110-4bd7-bc85-3dab6b471346 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:53.055770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960adc5-e1a0-405e-8823-c6048db68237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:53.056440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960adc5-e1a0-405e-8823-c6048db68237 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:53.056785 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960adc5-e1a0-405e-8823-c6048db68237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:53.057148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960adc5-e1a0-405e-8823-c6048db68237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:53.057865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960adc5-e1a0-405e-8823-c6048db68237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:53.059449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960adc5-e1a0-405e-8823-c6048db68237 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:53.069894 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1215/2410] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:45:53.074085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd54a64-57be-4c05-9ab9-e6990d86c93b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:53.084849 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd54a64-57be-4c05-9ab9-e6990d86c93b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:53.089473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238813f-8553-4d86-82da-e3e56703f8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:53.090118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238813f-8553-4d86-82da-e3e56703f8c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:53.091033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238813f-8553-4d86-82da-e3e56703f8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:53.091325 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238813f-8553-4d86-82da-e3e56703f8c6 admin admin] PATH_INFO: `/v3/projects/eea97f7a452d483e84990e50b3dd5495/users/665408a5a7aa4010bb643274dfb4274e/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:53.107237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238813f-8553-4d86-82da-e3e56703f8c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea97f7a452d483e84990e50b3dd5495, user_id=665408a5a7aa4010bb643274dfb4274e, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:53.109926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238813f-8553-4d86-82da-e3e56703f8c6 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:53.124007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1196/2411] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:53 2022] PUT /identity/v3/projects/eea97f7a452d483e84990e50b3dd5495/users/665408a5a7aa4010bb643274dfb4274e/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:45:53.128030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960adc5-e1a0-405e-8823-c6048db68237 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:53.138870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960adc5-e1a0-405e-8823-c6048db68237 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:53.200601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280f5bc-7b5b-4ea3-8720-bea5d952ab5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:53.201249 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280f5bc-7b5b-4ea3-8720-bea5d952ab5d admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:53.201534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280f5bc-7b5b-4ea3-8720-bea5d952ab5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:53.201808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280f5bc-7b5b-4ea3-8720-bea5d952ab5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:53.202446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280f5bc-7b5b-4ea3-8720-bea5d952ab5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:53.204033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280f5bc-7b5b-4ea3-8720-bea5d952ab5d admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:53.217321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1216/2412] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:45:53.218997 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238813f-8553-4d86-82da-e3e56703f8c6 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:53.233250 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238813f-8553-4d86-82da-e3e56703f8c6 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:53.238323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e88ea3-cfcc-44d3-8995-bb770fd82d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:53.238993 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e88ea3-cfcc-44d3-8995-bb770fd82d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:53.239270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e88ea3-cfcc-44d3-8995-bb770fd82d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:53.239541 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e88ea3-cfcc-44d3-8995-bb770fd82d1f admin admin] PATH_INFO: `/v3/projects/eea97f7a452d483e84990e50b3dd5495/users/665408a5a7aa4010bb643274dfb4274e/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:53.241528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e88ea3-cfcc-44d3-8995-bb770fd82d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea97f7a452d483e84990e50b3dd5495, user_id=665408a5a7aa4010bb643274dfb4274e, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:53.244161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e88ea3-cfcc-44d3-8995-bb770fd82d1f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:53.255864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1197/2413] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:45:53 2022] PUT /identity/v3/projects/eea97f7a452d483e84990e50b3dd5495/users/665408a5a7aa4010bb643274dfb4274e/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:53.266897 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6da8b4-f527-4fb6-85ed-03c8f6f30b0a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:53.267181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6da8b4-f527-4fb6-85ed-03c8f6f30b0a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:53.267456 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6da8b4-f527-4fb6-85ed-03c8f6f30b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:53.281609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:53.296848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-8f6da8b4-f527-4fb6-85ed-03c8f6f30b0a None None] MFA Rules not processed for user `665408a5a7aa4010bb643274dfb4274e`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:45:53.298269 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8f6da8b4-f527-4fb6-85ed-03c8f6f30b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:53.373657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1217/2414] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:45:53.385582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e88ea3-cfcc-44d3-8995-bb770fd82d1f admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:53.387841 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-06e88ea3-cfcc-44d3-8995-bb770fd82d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:53.446534 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e88ea3-cfcc-44d3-8995-bb770fd82d1f admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:53.450833 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162fbaa-0e98-43a6-8988-9681674aafbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:53.451432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162fbaa-0e98-43a6-8988-9681674aafbd service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:53.451705 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162fbaa-0e98-43a6-8988-9681674aafbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:53.451970 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162fbaa-0e98-43a6-8988-9681674aafbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:53.518099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162fbaa-0e98-43a6-8988-9681674aafbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:53.518099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162fbaa-0e98-43a6-8988-9681674aafbd service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:53.529580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1198/2415] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:55.088419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6da8b4-f527-4fb6-85ed-03c8f6f30b0a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:55.089820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8f6da8b4-f527-4fb6-85ed-03c8f6f30b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:45:55.102196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6da8b4-f527-4fb6-85ed-03c8f6f30b0a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:55.105725 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dfe25-b726-4f49-b50c-f6b4f64589c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:55.106365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dfe25-b726-4f49-b50c-f6b4f64589c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:55.106644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dfe25-b726-4f49-b50c-f6b4f64589c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:55.106939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dfe25-b726-4f49-b50c-f6b4f64589c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:55.162824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dfe25-b726-4f49-b50c-f6b4f64589c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:55.164249 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dfe25-b726-4f49-b50c-f6b4f64589c3 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:55.174399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1218/2416] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:55.763058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162fbaa-0e98-43a6-8988-9681674aafbd service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:55.771557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162fbaa-0e98-43a6-8988-9681674aafbd service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:55.779097 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3090b-3604-4a17-b5e0-0796116cbb42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:55.780316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3090b-3604-4a17-b5e0-0796116cbb42 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:55.780832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3090b-3604-4a17-b5e0-0796116cbb42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:55.781437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3090b-3604-4a17-b5e0-0796116cbb42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:55.797365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3090b-3604-4a17-b5e0-0796116cbb42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:55.798950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3090b-3604-4a17-b5e0-0796116cbb42 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:55.809662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1199/2417] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:45:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:45:59.759592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dfe25-b726-4f49-b50c-f6b4f64589c3 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:45:59.856863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:45:59.868125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dfe25-b726-4f49-b50c-f6b4f64589c3 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:45:59.872716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:45:59.873372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:59.873654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:59.873929 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] PATH_INFO: `/v3/users/1687d75e941d429a9b5069a2d8136813` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:45:59.875052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1687d75e941d429a9b5069a2d8136813)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:45:59.876964 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:45:59.930103 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1687d75e941d429a9b5069a2d8136813'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:59.943689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1687d75e941d429a9b5069a2d8136813'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:59.944340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1687d75e941d429a9b5069a2d8136813'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:59.952956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1687d75e941d429a9b5069a2d8136813'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:59.953646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1687d75e941d429a9b5069a2d8136813'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:45:59.967140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1219/2418] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:45:59 2022] DELETE /identity/v3/users/1687d75e941d429a9b5069a2d8136813 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:45:59.971617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e2705-3423-41ea-b2f0-ab6eea3e669f None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:45:59.972136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e2705-3423-41ea-b2f0-ab6eea3e669f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:45:59.972136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e2705-3423-41ea-b2f0-ab6eea3e669f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:00.017300 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:46:00.026184 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-502e2705-3423-41ea-b2f0-ab6eea3e669f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:46:00.027810 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-502e2705-3423-41ea-b2f0-ab6eea3e669f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:00.089506 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1200/2419] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:46:00.100480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:00.101928 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:00.177428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0366ab-470e-4030-9fba-171b116b2ea3 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:00.181237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f469c-4fa8-40f5-abbc-e15ed1a68085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:00.181991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f469c-4fa8-40f5-abbc-e15ed1a68085 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:00.182270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f469c-4fa8-40f5-abbc-e15ed1a68085 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:00.182544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f469c-4fa8-40f5-abbc-e15ed1a68085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:00.192728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f469c-4fa8-40f5-abbc-e15ed1a68085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:00.194254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f469c-4fa8-40f5-abbc-e15ed1a68085 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:00.208174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1220/2420] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:46:00.701293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e2705-3423-41ea-b2f0-ab6eea3e669f None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:00.703364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-502e2705-3423-41ea-b2f0-ab6eea3e669f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:00.718701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e2705-3423-41ea-b2f0-ab6eea3e669f None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:00.727741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:00.729071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:00.730709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:00.730709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] PATH_INFO: `/v3/projects/6a047cd61d4c4e3fa55bfa0084cbd167` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:00.733949 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a047cd61d4c4e3fa55bfa0084cbd167)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:00.742315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:00.804698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a047cd61d4c4e3fa55bfa0084cbd167', 'request_id': 'req-d52dfb53-63d6-47c6-a9ea-46adb875f6be'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:00.811135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a047cd61d4c4e3fa55bfa0084cbd167', 'request_id': 'req-d52dfb53-63d6-47c6-a9ea-46adb875f6be'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:00.818893 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a047cd61d4c4e3fa55bfa0084cbd167', 'request_id': 'req-d52dfb53-63d6-47c6-a9ea-46adb875f6be'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:00.831458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] The token cache is being invalidate because project 6a047cd61d4c4e3fa55bfa0084cbd167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:46:00.831751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:00.832469 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:00.834046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1201/2421] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:46:00 2022] DELETE /identity/v3/projects/6a047cd61d4c4e3fa55bfa0084cbd167 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:46:00.838161 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f469c-4fa8-40f5-abbc-e15ed1a68085 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:00.848111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f469c-4fa8-40f5-abbc-e15ed1a68085 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:00.902367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:46:00.910176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:00.910910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:00.911530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:00.911961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] PATH_INFO: `/v3/users/72962efcc4e44c6abd7d5bab45f8290a` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:00.913249 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72962efcc4e44c6abd7d5bab45f8290a)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:00.915162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:00.962794 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72962efcc4e44c6abd7d5bab45f8290a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:00.974946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72962efcc4e44c6abd7d5bab45f8290a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:00.975627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72962efcc4e44c6abd7d5bab45f8290a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:00.982781 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72962efcc4e44c6abd7d5bab45f8290a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:00.983440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72962efcc4e44c6abd7d5bab45f8290a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:00.997742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1221/2422] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:46:00 2022] DELETE /identity/v3/users/72962efcc4e44c6abd7d5bab45f8290a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:46:01.278557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:01.280061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:01.297284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dfb53-63d6-47c6-a9ea-46adb875f6be admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:01.358718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:01.359345 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:01.359621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:01.359948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] PATH_INFO: `/v3/projects/4036118fcf434626bc58ddf3800b3b65` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:01.361065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4036118fcf434626bc58ddf3800b3b65)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:01.363011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:01.397339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:46:01.411359 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4036118fcf434626bc58ddf3800b3b65', 'request_id': 'req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:01.417158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4036118fcf434626bc58ddf3800b3b65', 'request_id': 'req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:01.425247 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4036118fcf434626bc58ddf3800b3b65', 'request_id': 'req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:01.436516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] The token cache is being invalidate because project 4036118fcf434626bc58ddf3800b3b65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:46:01.436807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:01.437463 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:01.438950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1202/2423] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:46:01 2022] DELETE /identity/v3/projects/4036118fcf434626bc58ddf3800b3b65 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:46:01.442990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:01.444463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:01.457440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854408ab-79ef-408d-82d5-2cdb4a408989 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:01.514657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134cdbd-f320-4986-b328-a5fd1355997c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:01.515493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134cdbd-f320-4986-b328-a5fd1355997c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:01.515772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134cdbd-f320-4986-b328-a5fd1355997c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:01.516040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134cdbd-f320-4986-b328-a5fd1355997c admin admin] PATH_INFO: `/v3/users/665408a5a7aa4010bb643274dfb4274e` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:01.517191 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134cdbd-f320-4986-b328-a5fd1355997c admin admin] RBAC: Authorizing `identity:delete_user(user_id=665408a5a7aa4010bb643274dfb4274e)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:01.525010 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134cdbd-f320-4986-b328-a5fd1355997c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:01.572084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9134cdbd-f320-4986-b328-a5fd1355997c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '665408a5a7aa4010bb643274dfb4274e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:01.586011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9134cdbd-f320-4986-b328-a5fd1355997c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '665408a5a7aa4010bb643274dfb4274e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:01.586757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9134cdbd-f320-4986-b328-a5fd1355997c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '665408a5a7aa4010bb643274dfb4274e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:01.595520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9134cdbd-f320-4986-b328-a5fd1355997c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '665408a5a7aa4010bb643274dfb4274e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:01.596203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9134cdbd-f320-4986-b328-a5fd1355997c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '665408a5a7aa4010bb643274dfb4274e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:01.600082 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:46:01.614036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1222/2424] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:46:01 2022] DELETE /identity/v3/users/665408a5a7aa4010bb643274dfb4274e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:46:01.895590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:01.897063 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:01.912786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc30e0-2bbd-4fc0-9859-3b80e3992d53 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:01.981786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:01.982508 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:01.982857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:01.983126 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9 admin admin] PATH_INFO: `/v3/projects/eea97f7a452d483e84990e50b3dd5495` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:01.984244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eea97f7a452d483e84990e50b3dd5495)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:01.986196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:02.034508 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eea97f7a452d483e84990e50b3dd5495', 'request_id': 'req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:02.040066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eea97f7a452d483e84990e50b3dd5495', 'request_id': 'req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:02.046982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eea97f7a452d483e84990e50b3dd5495', 'request_id': 'req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:02.057931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9 admin admin] The token cache is being invalidate because project eea97f7a452d483e84990e50b3dd5495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:46:02.058215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:02.058861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-db8fbc2f-12ea-4ebb-a913-bae6f09a28b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:02.060319 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1203/2425] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:46:01 2022] DELETE /identity/v3/projects/eea97f7a452d483e84990e50b3dd5495 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:46:02.065388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64f6ae-7234-43f5-b5f7-ca5c7da338ac None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:02.065875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64f6ae-7234-43f5-b5f7-ca5c7da338ac None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:02.065968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64f6ae-7234-43f5-b5f7-ca5c7da338ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:02.115030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-db64f6ae-7234-43f5-b5f7-ca5c7da338ac None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:46:02.116360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-db64f6ae-7234-43f5-b5f7-ca5c7da338ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:02.176572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1223/2426] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:46:02.184965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aaaf2-e74b-45ca-a4be-cd48f7f9e799 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:02.185285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aaaf2-e74b-45ca-a4be-cd48f7f9e799 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:02.185562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aaaf2-e74b-45ca-a4be-cd48f7f9e799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:02.207435 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-018aaaf2-e74b-45ca-a4be-cd48f7f9e799 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:46:02.208885 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-018aaaf2-e74b-45ca-a4be-cd48f7f9e799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:02.217492 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1204/2427] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:46:02.222698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64f6ae-7234-43f5-b5f7-ca5c7da338ac None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:02.233111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64f6ae-7234-43f5-b5f7-ca5c7da338ac None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:02.237761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06407394-ad34-4085-94e5-a96e02c3b058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:02.238375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06407394-ad34-4085-94e5-a96e02c3b058 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:02.238655 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06407394-ad34-4085-94e5-a96e02c3b058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:02.238928 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06407394-ad34-4085-94e5-a96e02c3b058 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:02.239576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06407394-ad34-4085-94e5-a96e02c3b058 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:46:02.239881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06407394-ad34-4085-94e5-a96e02c3b058 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:02.241546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06407394-ad34-4085-94e5-a96e02c3b058 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:02.254531 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1224/2428] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:46:02.258569 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aaaf2-e74b-45ca-a4be-cd48f7f9e799 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:02.268461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aaaf2-e74b-45ca-a4be-cd48f7f9e799 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:02.273550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c961715-f846-46da-981b-d6deb6776e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:02.274159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c961715-f846-46da-981b-d6deb6776e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:02.274431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c961715-f846-46da-981b-d6deb6776e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:02.274699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c961715-f846-46da-981b-d6deb6776e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:02.275332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c961715-f846-46da-981b-d6deb6776e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:02.277232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c961715-f846-46da-981b-d6deb6776e5d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:02.308546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1205/2429] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:46:02 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:46:02.313712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06407394-ad34-4085-94e5-a96e02c3b058 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:02.324175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06407394-ad34-4085-94e5-a96e02c3b058 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:02.386320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40167ab3-a905-4bf8-99dd-3b982cafd7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:02.387566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40167ab3-a905-4bf8-99dd-3b982cafd7ca admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:02.387850 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40167ab3-a905-4bf8-99dd-3b982cafd7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:02.388120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40167ab3-a905-4bf8-99dd-3b982cafd7ca admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:02.388745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40167ab3-a905-4bf8-99dd-3b982cafd7ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:02.390710 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40167ab3-a905-4bf8-99dd-3b982cafd7ca admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:02.411627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1225/2430] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:46:02 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:46:02.415802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c961715-f846-46da-981b-d6deb6776e5d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:02.426436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c961715-f846-46da-981b-d6deb6776e5d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:02.431052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c63588-93fe-4e2c-9e70-16529a4a296b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:02.431646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c63588-93fe-4e2c-9e70-16529a4a296b admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:02.431925 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c63588-93fe-4e2c-9e70-16529a4a296b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:02.432197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c63588-93fe-4e2c-9e70-16529a4a296b admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:02.432848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c63588-93fe-4e2c-9e70-16529a4a296b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:02.434967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c63588-93fe-4e2c-9e70-16529a4a296b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:02.444944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1206/2431] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:46:02.448814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40167ab3-a905-4bf8-99dd-3b982cafd7ca admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:02.459361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40167ab3-a905-4bf8-99dd-3b982cafd7ca admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:02.463941 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10871cf2-1448-432b-87dc-78da0070b16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:02.464530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10871cf2-1448-432b-87dc-78da0070b16d admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:02.464807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10871cf2-1448-432b-87dc-78da0070b16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:02.465077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10871cf2-1448-432b-87dc-78da0070b16d admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:02.465742 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10871cf2-1448-432b-87dc-78da0070b16d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:02.467304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10871cf2-1448-432b-87dc-78da0070b16d admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:02.478171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1226/2432] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:46:02.482524 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c63588-93fe-4e2c-9e70-16529a4a296b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:02.492430 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c63588-93fe-4e2c-9e70-16529a4a296b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:02.497645 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44092edf-f273-4286-8543-2a0ffdc8219b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:02.498420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44092edf-f273-4286-8543-2a0ffdc8219b admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:02.498735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44092edf-f273-4286-8543-2a0ffdc8219b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:02.499013 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44092edf-f273-4286-8543-2a0ffdc8219b admin admin] PATH_INFO: `/v3/projects/bfe1975eab4d4dad9aab52c90493bf7d/users/5cae6744964d4cabaf0ef00a23057c85/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:02.502094 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:46:02.516314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44092edf-f273-4286-8543-2a0ffdc8219b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe1975eab4d4dad9aab52c90493bf7d, user_id=5cae6744964d4cabaf0ef00a23057c85, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:02.519084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44092edf-f273-4286-8543-2a0ffdc8219b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:02.533629 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1207/2433] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:46:02 2022] PUT /identity/v3/projects/bfe1975eab4d4dad9aab52c90493bf7d/users/5cae6744964d4cabaf0ef00a23057c85/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:46:02.538997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87292b3f-1a16-406d-b069-fadd65fc38b7 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:02.539273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87292b3f-1a16-406d-b069-fadd65fc38b7 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:02.539590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87292b3f-1a16-406d-b069-fadd65fc38b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:02.559812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-87292b3f-1a16-406d-b069-fadd65fc38b7 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:46:02.560932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-87292b3f-1a16-406d-b069-fadd65fc38b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:02.590876 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:46:02.621051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1227/2434] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:46:02.631292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44092edf-f273-4286-8543-2a0ffdc8219b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:02.632995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-44092edf-f273-4286-8543-2a0ffdc8219b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:02.689952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44092edf-f273-4286-8543-2a0ffdc8219b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:02.698815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6988f19-7422-46df-812f-bddc97449275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:02.699955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988f19-7422-46df-812f-bddc97449275 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:02.700230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988f19-7422-46df-812f-bddc97449275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:02.700497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6988f19-7422-46df-812f-bddc97449275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:02.710608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6988f19-7422-46df-812f-bddc97449275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:02.712026 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6988f19-7422-46df-812f-bddc97449275 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:02.722245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1208/2435] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:46:03.384156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a083737-89b6-416d-8d20-3aa2b8ba1d05 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:03.384700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a083737-89b6-416d-8d20-3aa2b8ba1d05 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:03.384815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a083737-89b6-416d-8d20-3aa2b8ba1d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:03.412716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2a083737-89b6-416d-8d20-3aa2b8ba1d05 None None] MFA Rules not processed for user `5cae6744964d4cabaf0ef00a23057c85`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:46:03.414353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2a083737-89b6-416d-8d20-3aa2b8ba1d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:03.482771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1228/2436] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:46:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:46:03.493732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6988f19-7422-46df-812f-bddc97449275 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:03.495218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b6988f19-7422-46df-812f-bddc97449275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:03.562079 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6988f19-7422-46df-812f-bddc97449275 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:03.565802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b05771-ff73-4348-8a2c-6ce66ddeb46f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:03.566416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b05771-ff73-4348-8a2c-6ce66ddeb46f service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:03.566695 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b05771-ff73-4348-8a2c-6ce66ddeb46f service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:03.566967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b05771-ff73-4348-8a2c-6ce66ddeb46f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:03.575626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b05771-ff73-4348-8a2c-6ce66ddeb46f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:03.577114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b05771-ff73-4348-8a2c-6ce66ddeb46f service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:03.589901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1209/2437] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:46:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:46:04.991317 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bf43d-f6ec-4e3b-9400-3d5c4af4dc5e None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:04.991839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bf43d-f6ec-4e3b-9400-3d5c4af4dc5e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:04.991839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bf43d-f6ec-4e3b-9400-3d5c4af4dc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:05.011777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-4e4bf43d-f6ec-4e3b-9400-3d5c4af4dc5e None None] MFA Rules not processed for user `5cae6744964d4cabaf0ef00a23057c85`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:46:05.013521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4e4bf43d-f6ec-4e3b-9400-3d5c4af4dc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:05.020303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1229/2438] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:46:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:46:12.771112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b05771-ff73-4348-8a2c-6ce66ddeb46f service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:12.772581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-92b05771-ff73-4348-8a2c-6ce66ddeb46f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:12.878105 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b05771-ff73-4348-8a2c-6ce66ddeb46f service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:12.884510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888f2ff-1e32-42c0-abc4-cd45af649518 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:12.885262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888f2ff-1e32-42c0-abc4-cd45af649518 service placement] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:12.885616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888f2ff-1e32-42c0-abc4-cd45af649518 service placement] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:12.885967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888f2ff-1e32-42c0-abc4-cd45af649518 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:12.887881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6888f2ff-1e32-42c0-abc4-cd45af649518 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:12.900273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888f2ff-1e32-42c0-abc4-cd45af649518 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:12.902527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888f2ff-1e32-42c0-abc4-cd45af649518 service placement] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:12.909255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:46:12.914944 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1210/2439] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:46:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:46:15.005254 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=46 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:46:31.755049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4bf43d-f6ec-4e3b-9400-3d5c4af4dc5e None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:31.774248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4bf43d-f6ec-4e3b-9400-3d5c4af4dc5e None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:31.777852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5670b-6ab3-4f8d-82d7-9a9a8521e767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:31.778387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5670b-6ab3-4f8d-82d7-9a9a8521e767 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:31.778662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5670b-6ab3-4f8d-82d7-9a9a8521e767 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:31.778931 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5670b-6ab3-4f8d-82d7-9a9a8521e767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:31.780635 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a0f5670b-6ab3-4f8d-82d7-9a9a8521e767 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:31.843634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5670b-6ab3-4f8d-82d7-9a9a8521e767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:31.845083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5670b-6ab3-4f8d-82d7-9a9a8521e767 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:31.854527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1230/2440] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:46:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:46:37.984375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99685a-114d-4c5a-a091-73b9d3414ddf None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:37.984840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99685a-114d-4c5a-a091-73b9d3414ddf None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:37.984934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99685a-114d-4c5a-a091-73b9d3414ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:38.005926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-eb99685a-114d-4c5a-a091-73b9d3414ddf None None] MFA Rules not processed for user `7e3b9ceb95374f1dbf356122c8ed463c`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:46:38.007252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-eb99685a-114d-4c5a-a091-73b9d3414ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:38.048454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5670b-6ab3-4f8d-82d7-9a9a8521e767 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:38.057165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5670b-6ab3-4f8d-82d7-9a9a8521e767 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:38.060894 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ecca4-e2fd-4d9a-bca8-6522ac4320eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:38.061530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ecca4-e2fd-4d9a-bca8-6522ac4320eb service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:38.061812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ecca4-e2fd-4d9a-bca8-6522ac4320eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:38.062106 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ecca4-e2fd-4d9a-bca8-6522ac4320eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:38.063919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ac3ecca4-e2fd-4d9a-bca8-6522ac4320eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:38.076020 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ecca4-e2fd-4d9a-bca8-6522ac4320eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:38.077992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ecca4-e2fd-4d9a-bca8-6522ac4320eb service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:38.081451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1211/2441] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:46:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:46:38.088860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1231/2442] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:46:42.304993 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99685a-114d-4c5a-a091-73b9d3414ddf None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:42.306604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-eb99685a-114d-4c5a-a091-73b9d3414ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:42.334432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99685a-114d-4c5a-a091-73b9d3414ddf None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:42.350877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbdffb-05db-4c4d-b47d-381685da0929 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:42.352274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbdffb-05db-4c4d-b47d-381685da0929 service placement] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:42.353065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbdffb-05db-4c4d-b47d-381685da0929 service placement] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:42.353542 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbdffb-05db-4c4d-b47d-381685da0929 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:42.359928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1edbdffb-05db-4c4d-b47d-381685da0929 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:42.381610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbdffb-05db-4c4d-b47d-381685da0929 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:42.383112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbdffb-05db-4c4d-b47d-381685da0929 service placement] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:42.394264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1212/2443] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:46:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:46:46.494663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ecca4-e2fd-4d9a-bca8-6522ac4320eb service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:46.496260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ac3ecca4-e2fd-4d9a-bca8-6522ac4320eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:46.517687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ecca4-e2fd-4d9a-bca8-6522ac4320eb service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:46.517687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:46.517687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:46.517687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:46.518202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] PATH_INFO: `/v3/users/4f250207e16c4515bd9beb9ca4b1fde6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:46.521519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f250207e16c4515bd9beb9ca4b1fde6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:46.523609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:46.577796 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f250207e16c4515bd9beb9ca4b1fde6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:46.592472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f250207e16c4515bd9beb9ca4b1fde6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:46.593265 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f250207e16c4515bd9beb9ca4b1fde6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:46.602478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f250207e16c4515bd9beb9ca4b1fde6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:46.602478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f250207e16c4515bd9beb9ca4b1fde6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:46.617154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1232/2444] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:46:46 2022] DELETE /identity/v3/users/4f250207e16c4515bd9beb9ca4b1fde6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:46:46.823497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbdffb-05db-4c4d-b47d-381685da0929 service placement] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:46.835358 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbdffb-05db-4c4d-b47d-381685da0929 service placement] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:46.904916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:46.905639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:46.905922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:46.906192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] PATH_INFO: `/v3/projects/c507f95ba3c34cc789ee8903b40c346d` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:46.907461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c507f95ba3c34cc789ee8903b40c346d)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:46.909414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:46.956723 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c507f95ba3c34cc789ee8903b40c346d', 'request_id': 'req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:46.962196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c507f95ba3c34cc789ee8903b40c346d', 'request_id': 'req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:46.969766 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c507f95ba3c34cc789ee8903b40c346d', 'request_id': 'req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:46.980782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] The token cache is being invalidate because project c507f95ba3c34cc789ee8903b40c346d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:46:46.981074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:46.981731 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:46.983771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1213/2445] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:46:46 2022] DELETE /identity/v3/projects/c507f95ba3c34cc789ee8903b40c346d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:46:46.987846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:46.989277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:47.003157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ad53f-72d7-4e49-af74-f5b4dfc167d8 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:47.051054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:46:47.068292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e040ee-fa91-4f9a-89d3-f7b45bbeb13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:47.069308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e040ee-fa91-4f9a-89d3-f7b45bbeb13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:47.069738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e040ee-fa91-4f9a-89d3-f7b45bbeb13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:47.070154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e040ee-fa91-4f9a-89d3-f7b45bbeb13e admin admin] PATH_INFO: `/v3/users/ab78cd046b2742de85d5d11501b78572` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:47.071816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e040ee-fa91-4f9a-89d3-f7b45bbeb13e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab78cd046b2742de85d5d11501b78572)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:47.074631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e040ee-fa91-4f9a-89d3-f7b45bbeb13e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:47.127217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-33e040ee-fa91-4f9a-89d3-f7b45bbeb13e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab78cd046b2742de85d5d11501b78572'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:47.139677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-33e040ee-fa91-4f9a-89d3-f7b45bbeb13e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab78cd046b2742de85d5d11501b78572'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:47.140355 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-33e040ee-fa91-4f9a-89d3-f7b45bbeb13e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab78cd046b2742de85d5d11501b78572'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:47.147153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-33e040ee-fa91-4f9a-89d3-f7b45bbeb13e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab78cd046b2742de85d5d11501b78572'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:47.147770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-33e040ee-fa91-4f9a-89d3-f7b45bbeb13e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab78cd046b2742de85d5d11501b78572'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:47.160356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1233/2446] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:46:46 2022] DELETE /identity/v3/users/ab78cd046b2742de85d5d11501b78572 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:46:47.336189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:46:47.338655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:46:47.357420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b6d7e-6d8a-4f83-b7b1-166ff2fa962e admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:46:47.386789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:46:47.436772 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:46:47.437475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:46:47.437759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:46:47.438033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] PATH_INFO: `/v3/projects/f6dac82b091e40b38f9fd37de64e1566` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:46:47.439165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6dac82b091e40b38f9fd37de64e1566)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:46:47.441112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:46:47.488982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6dac82b091e40b38f9fd37de64e1566', 'request_id': 'req-4056329c-bb38-45a5-bbaf-d14969a067c8'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:47.494525 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6dac82b091e40b38f9fd37de64e1566', 'request_id': 'req-4056329c-bb38-45a5-bbaf-d14969a067c8'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:47.502515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6dac82b091e40b38f9fd37de64e1566', 'request_id': 'req-4056329c-bb38-45a5-bbaf-d14969a067c8'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:47.513736 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] The token cache is being invalidate because project f6dac82b091e40b38f9fd37de64e1566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:46:47.513988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:47.514629 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:46:47.516107 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1214/2447] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:46:47 2022] DELETE /identity/v3/projects/f6dac82b091e40b38f9fd37de64e1566 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:46:57.156991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:46:57.510699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:47:29.975227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1d51d-51d2-4a90-a78f-1c84ac789947 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:47:29.976397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1d51d-51d2-4a90-a78f-1c84ac789947 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:47:29.976397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1d51d-51d2-4a90-a78f-1c84ac789947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:47:29.995043 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-aff1d51d-51d2-4a90-a78f-1c84ac789947 None None] MFA Rules not processed for user `5cae6744964d4cabaf0ef00a23057c85`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:47:29.996315 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-aff1d51d-51d2-4a90-a78f-1c84ac789947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:47:30.050836 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1234/2448] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:47:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:47:40.043533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=25 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:47:57.776553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:47:57.777905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:47:57.847310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056329c-bb38-45a5-bbaf-d14969a067c8 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:47:57.851340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700e30c-15f4-4eaf-ac52-6613404347bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:47:57.851941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700e30c-15f4-4eaf-ac52-6613404347bd service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:47:57.852211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700e30c-15f4-4eaf-ac52-6613404347bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:47:57.852476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700e30c-15f4-4eaf-ac52-6613404347bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:47:57.854192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3700e30c-15f4-4eaf-ac52-6613404347bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:47:57.946374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700e30c-15f4-4eaf-ac52-6613404347bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:47:57.947814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700e30c-15f4-4eaf-ac52-6613404347bd service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:47:57.968194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1215/2449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:47:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:47:59.148597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1d51d-51d2-4a90-a78f-1c84ac789947 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:47:59.158447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1d51d-51d2-4a90-a78f-1c84ac789947 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:47:59.162133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3646ad5-f15d-45e4-9c28-c6ed6bbc23e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:47:59.162735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3646ad5-f15d-45e4-9c28-c6ed6bbc23e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:47:59.163019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3646ad5-f15d-45e4-9c28-c6ed6bbc23e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:47:59.163481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3646ad5-f15d-45e4-9c28-c6ed6bbc23e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:47:59.165919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c3646ad5-f15d-45e4-9c28-c6ed6bbc23e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:47:59.181746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3646ad5-f15d-45e4-9c28-c6ed6bbc23e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:47:59.183184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3646ad5-f15d-45e4-9c28-c6ed6bbc23e5 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:47:59.196077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1235/2450] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:47:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:48:04.293106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56e9ba-dca0-4635-b5b3-9f08d2e57503 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:48:04.293628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56e9ba-dca0-4635-b5b3-9f08d2e57503 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:48:04.293729 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56e9ba-dca0-4635-b5b3-9f08d2e57503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:48:04.324434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-eb56e9ba-dca0-4635-b5b3-9f08d2e57503 None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:48:04.325726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-eb56e9ba-dca0-4635-b5b3-9f08d2e57503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:48:04.375771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:48:04.383427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1216/2451] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:48:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:48:04.439056 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3646ad5-f15d-45e4-9c28-c6ed6bbc23e5 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:48:04.440539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c3646ad5-f15d-45e4-9c28-c6ed6bbc23e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:48:04.509547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3646ad5-f15d-45e4-9c28-c6ed6bbc23e5 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:48:04.514659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307301d-8aa7-4df7-8ac2-cd3aa3120c86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:48:04.515273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307301d-8aa7-4df7-8ac2-cd3aa3120c86 service swift] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:48:04.515552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307301d-8aa7-4df7-8ac2-cd3aa3120c86 service swift] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:48:04.515826 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307301d-8aa7-4df7-8ac2-cd3aa3120c86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:48:04.525304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307301d-8aa7-4df7-8ac2-cd3aa3120c86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:48:04.527494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307301d-8aa7-4df7-8ac2-cd3aa3120c86 service swift] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:48:04.535234 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1236/2452] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:48:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:48:14.531823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=38 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:48:24.539319 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507c8c1-673d-4613-ac22-5fe02621a7b8 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:48:24.539861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507c8c1-673d-4613-ac22-5fe02621a7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:48:24.539861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507c8c1-673d-4613-ac22-5fe02621a7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:48:24.598066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-2507c8c1-673d-4613-ac22-5fe02621a7b8 None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:48:24.599329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2507c8c1-673d-4613-ac22-5fe02621a7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:48:24.618602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1217/2453] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:48:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:48:24.636817 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307301d-8aa7-4df7-8ac2-cd3aa3120c86 service swift] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:48:24.648067 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307301d-8aa7-4df7-8ac2-cd3aa3120c86 service swift] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:48:24.653396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173c18a-4615-4321-8d1c-ab95f96d1f37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:48:24.654001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173c18a-4615-4321-8d1c-ab95f96d1f37 service swift] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:48:24.654280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173c18a-4615-4321-8d1c-ab95f96d1f37 service swift] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:48:24.654582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173c18a-4615-4321-8d1c-ab95f96d1f37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:48:24.663517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173c18a-4615-4321-8d1c-ab95f96d1f37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:48:24.665138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173c18a-4615-4321-8d1c-ab95f96d1f37 service swift] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:48:24.674255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1237/2454] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:48:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 08:48:34.614167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=18 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:48:34.670261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=6 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:48:43.297041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507c8c1-673d-4613-ac22-5fe02621a7b8 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:48:43.298948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2507c8c1-673d-4613-ac22-5fe02621a7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:48:43.367093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507c8c1-673d-4613-ac22-5fe02621a7b8 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:48:43.372017 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:48:43.372687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:48:43.372994 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:48:43.373470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] PATH_INFO: `/v3/users/5cae6744964d4cabaf0ef00a23057c85` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:48:43.374500 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cae6744964d4cabaf0ef00a23057c85)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:48:43.376572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:48:43.426128 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cae6744964d4cabaf0ef00a23057c85'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:48:43.433626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cae6744964d4cabaf0ef00a23057c85'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:48:43.434396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cae6744964d4cabaf0ef00a23057c85'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:48:43.435015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cae6744964d4cabaf0ef00a23057c85'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:48:43.442299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cae6744964d4cabaf0ef00a23057c85'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:48:43.475168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1218/2455] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:48:43 2022] DELETE /identity/v3/users/5cae6744964d4cabaf0ef00a23057c85 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:48:43.660154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173c18a-4615-4321-8d1c-ab95f96d1f37 service swift] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:48:43.671666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173c18a-4615-4321-8d1c-ab95f96d1f37 service swift] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:48:43.737182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:48:43.737831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:48:43.738117 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:48:43.738533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4 admin admin] PATH_INFO: `/v3/projects/bfe1975eab4d4dad9aab52c90493bf7d` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:48:43.739933 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe1975eab4d4dad9aab52c90493bf7d)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:48:43.742411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:48:43.800753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe1975eab4d4dad9aab52c90493bf7d', 'request_id': 'req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:48:43.806524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe1975eab4d4dad9aab52c90493bf7d', 'request_id': 'req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:48:43.812883 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe1975eab4d4dad9aab52c90493bf7d', 'request_id': 'req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:48:43.834221 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4 admin admin] The token cache is being invalidate because project bfe1975eab4d4dad9aab52c90493bf7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:48:43.834522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:48:43.835214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f7c864d5-63b8-40cd-a34a-e21e5ba028a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:48:43.836904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1238/2456] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:48:43 2022] DELETE /identity/v3/projects/bfe1975eab4d4dad9aab52c90493bf7d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:48:47.927121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:48:47.928460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:48:47.985759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465d379-41f4-4280-90e9-f454afb3a520 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:48:48.016977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:48:48.036629 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098964e-8383-4f95-8a37-0ad4e692620e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:48:48.037260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098964e-8383-4f95-8a37-0ad4e692620e service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:48:48.037533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098964e-8383-4f95-8a37-0ad4e692620e service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:48:48.037799 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098964e-8383-4f95-8a37-0ad4e692620e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:48:48.039526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f098964e-8383-4f95-8a37-0ad4e692620e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:48:48.048850 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098964e-8383-4f95-8a37-0ad4e692620e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:48:48.050283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098964e-8383-4f95-8a37-0ad4e692620e service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:48:48.058008 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1219/2457] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:48:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:48:50.115691 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1fa4f-9253-4135-b8f2-d0388511c77a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:48:50.116156 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1fa4f-9253-4135-b8f2-d0388511c77a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:48:50.116255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1fa4f-9253-4135-b8f2-d0388511c77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:48:50.136128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2ea1fa4f-9253-4135-b8f2-d0388511c77a None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:48:50.137373 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2ea1fa4f-9253-4135-b8f2-d0388511c77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:48:50.194374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1239/2458] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 08:48:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:48:50.206616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098964e-8383-4f95-8a37-0ad4e692620e service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:48:50.208037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f098964e-8383-4f95-8a37-0ad4e692620e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:48:50.303806 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098964e-8383-4f95-8a37-0ad4e692620e service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:48:50.308892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d8640-0325-4478-a497-51ecad938499 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:48:50.309505 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d8640-0325-4478-a497-51ecad938499 service swift] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:48:50.309786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d8640-0325-4478-a497-51ecad938499 service swift] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:48:50.310071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d8640-0325-4478-a497-51ecad938499 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:48:50.318125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d8640-0325-4478-a497-51ecad938499 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:48:50.319699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d8640-0325-4478-a497-51ecad938499 service swift] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:48:50.327690 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1220/2459] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:48:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 08:49:00.186977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:00.324416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=51 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:06.070786 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1fa4f-9253-4135-b8f2-d0388511c77a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:06.072302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2ea1fa4f-9253-4135-b8f2-d0388511c77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:06.084352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1fa4f-9253-4135-b8f2-d0388511c77a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:06.088947 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd0413-ae0f-4bf2-ab42-51736d3d2896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:06.089606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd0413-ae0f-4bf2-ab42-51736d3d2896 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:06.089895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd0413-ae0f-4bf2-ab42-51736d3d2896 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:06.090169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd0413-ae0f-4bf2-ab42-51736d3d2896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:06.091886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-93bd0413-ae0f-4bf2-ab42-51736d3d2896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:06.156835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd0413-ae0f-4bf2-ab42-51736d3d2896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:06.158296 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd0413-ae0f-4bf2-ab42-51736d3d2896 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:06.168477 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1240/2460] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:09.296783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d8640-0325-4478-a497-51ecad938499 service swift] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:09.298559 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-462d8640-0325-4478-a497-51ecad938499 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:09.318971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d8640-0325-4478-a497-51ecad938499 service swift] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:09.323412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71238f54-a5d3-46d4-9bb6-45eb08584e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:09.323983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71238f54-a5d3-46d4-9bb6-45eb08584e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:09.324271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71238f54-a5d3-46d4-9bb6-45eb08584e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:09.324530 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71238f54-a5d3-46d4-9bb6-45eb08584e03 admin admin] PATH_INFO: `/v3/users/fe0fcdeafae645a582afcee3dde79ebc` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:09.339998 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71238f54-a5d3-46d4-9bb6-45eb08584e03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe0fcdeafae645a582afcee3dde79ebc)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:09.341982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71238f54-a5d3-46d4-9bb6-45eb08584e03 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:09.399476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-71238f54-a5d3-46d4-9bb6-45eb08584e03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe0fcdeafae645a582afcee3dde79ebc'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:09.406552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-71238f54-a5d3-46d4-9bb6-45eb08584e03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe0fcdeafae645a582afcee3dde79ebc'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:09.407255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-71238f54-a5d3-46d4-9bb6-45eb08584e03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe0fcdeafae645a582afcee3dde79ebc'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:09.407881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-71238f54-a5d3-46d4-9bb6-45eb08584e03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe0fcdeafae645a582afcee3dde79ebc'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:09.415403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-71238f54-a5d3-46d4-9bb6-45eb08584e03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe0fcdeafae645a582afcee3dde79ebc'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:09.433264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1221/2461] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:49:09 2022] DELETE /identity/v3/users/fe0fcdeafae645a582afcee3dde79ebc => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:09.611226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd0413-ae0f-4bf2-ab42-51736d3d2896 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:09.622705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd0413-ae0f-4bf2-ab42-51736d3d2896 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:09.683236 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:09.683872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:09.684151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:09.684426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7 admin admin] PATH_INFO: `/v3/projects/d3e65d28e5664025978902943452f338` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:09.696005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e65d28e5664025978902943452f338)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:09.697977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:09.751214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e65d28e5664025978902943452f338', 'request_id': 'req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:09.758138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e65d28e5664025978902943452f338', 'request_id': 'req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:09.767090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e65d28e5664025978902943452f338', 'request_id': 'req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:09.781197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7 admin admin] The token cache is being invalidate because project d3e65d28e5664025978902943452f338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:49:09.781520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:09.782160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-c4e4b5fc-7978-4e32-ab27-7d1762cad0c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:09.783792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1241/2462] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:49:09 2022] DELETE /identity/v3/projects/d3e65d28e5664025978902943452f338 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:09.789919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cb73c-c365-4ae5-a0cd-5318bf6fc92a None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:09.790233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cb73c-c365-4ae5-a0cd-5318bf6fc92a None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:09.790512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cb73c-c365-4ae5-a0cd-5318bf6fc92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:09.810055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ab6cb73c-c365-4ae5-a0cd-5318bf6fc92a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:09.811253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ab6cb73c-c365-4ae5-a0cd-5318bf6fc92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:09.870839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1222/2463] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:09.881358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ce9a8-4ec5-4031-94bf-270be600fd11 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:09.881671 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ce9a8-4ec5-4031-94bf-270be600fd11 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:09.881950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ce9a8-4ec5-4031-94bf-270be600fd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:09.905180 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-c13ce9a8-4ec5-4031-94bf-270be600fd11 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:09.906398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c13ce9a8-4ec5-4031-94bf-270be600fd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:09.914206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1242/2464] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:09.919399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cb73c-c365-4ae5-a0cd-5318bf6fc92a None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:09.928638 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cb73c-c365-4ae5-a0cd-5318bf6fc92a None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:09.934724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9038d74-2e42-4bf1-a88a-a0cf76445141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:09.935322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9038d74-2e42-4bf1-a88a-a0cf76445141 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:09.935595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9038d74-2e42-4bf1-a88a-a0cf76445141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:09.935862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9038d74-2e42-4bf1-a88a-a0cf76445141 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:09.936551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9038d74-2e42-4bf1-a88a-a0cf76445141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:49:09.936847 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9038d74-2e42-4bf1-a88a-a0cf76445141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:09.938500 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9038d74-2e42-4bf1-a88a-a0cf76445141 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:09.950602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1223/2465] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:49:09.955052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ce9a8-4ec5-4031-94bf-270be600fd11 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:09.965485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ce9a8-4ec5-4031-94bf-270be600fd11 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:09.970177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403427a-eda7-4994-866e-a398dcd92482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:09.970777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403427a-eda7-4994-866e-a398dcd92482 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:09.971073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403427a-eda7-4994-866e-a398dcd92482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:09.971345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8403427a-eda7-4994-866e-a398dcd92482 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:09.972014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403427a-eda7-4994-866e-a398dcd92482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:09.973943 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8403427a-eda7-4994-866e-a398dcd92482 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:09.979222 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:10.020863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1243/2466] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:49:09 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:10.025033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9038d74-2e42-4bf1-a88a-a0cf76445141 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:10.035155 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9038d74-2e42-4bf1-a88a-a0cf76445141 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:10.080828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:10.098940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25a745-8c64-4704-91cf-6fa31c604bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:10.100274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25a745-8c64-4704-91cf-6fa31c604bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:10.100921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25a745-8c64-4704-91cf-6fa31c604bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:10.101240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25a745-8c64-4704-91cf-6fa31c604bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:10.101876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25a745-8c64-4704-91cf-6fa31c604bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:10.103809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25a745-8c64-4704-91cf-6fa31c604bc1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:10.133184 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1224/2467] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:49:10 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:10.137016 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403427a-eda7-4994-866e-a398dcd92482 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:10.148543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8403427a-eda7-4994-866e-a398dcd92482 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:10.153433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674803dc-468b-47b5-927f-28ebc5630adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:10.154130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674803dc-468b-47b5-927f-28ebc5630adb admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:10.154416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674803dc-468b-47b5-927f-28ebc5630adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:10.154696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674803dc-468b-47b5-927f-28ebc5630adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:10.155351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674803dc-468b-47b5-927f-28ebc5630adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:10.156975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674803dc-468b-47b5-927f-28ebc5630adb admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:10.167351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1244/2468] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:49:10.171391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25a745-8c64-4704-91cf-6fa31c604bc1 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:10.182570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25a745-8c64-4704-91cf-6fa31c604bc1 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:10.187032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1013e-9d3c-4d28-b096-13a23b584cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:10.187634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1013e-9d3c-4d28-b096-13a23b584cce admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:10.187919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1013e-9d3c-4d28-b096-13a23b584cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:10.188195 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1013e-9d3c-4d28-b096-13a23b584cce admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:10.188835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1013e-9d3c-4d28-b096-13a23b584cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:10.190580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1013e-9d3c-4d28-b096-13a23b584cce admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:10.201963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1225/2469] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:49:10.206186 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674803dc-468b-47b5-927f-28ebc5630adb admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:10.216756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674803dc-468b-47b5-927f-28ebc5630adb admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:10.221402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33169cf1-eb99-4015-bd0c-64f7a19bf98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:10.222049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33169cf1-eb99-4015-bd0c-64f7a19bf98d admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:10.222326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33169cf1-eb99-4015-bd0c-64f7a19bf98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:10.222599 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33169cf1-eb99-4015-bd0c-64f7a19bf98d admin admin] PATH_INFO: `/v3/projects/a61f5e00c9434810994bca8eb23255dc/users/7a7b54e21f3142dfae74495c8bcb6511/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:10.239338 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33169cf1-eb99-4015-bd0c-64f7a19bf98d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a61f5e00c9434810994bca8eb23255dc, user_id=7a7b54e21f3142dfae74495c8bcb6511, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:10.242515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33169cf1-eb99-4015-bd0c-64f7a19bf98d admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:10.259633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1245/2470] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:49:10 2022] PUT /identity/v3/projects/a61f5e00c9434810994bca8eb23255dc/users/7a7b54e21f3142dfae74495c8bcb6511/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:10.269962 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102b959-a1cf-456f-bef8-3405fe76ffec None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:10.270246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102b959-a1cf-456f-bef8-3405fe76ffec None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:10.270643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102b959-a1cf-456f-bef8-3405fe76ffec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:10.302173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-2102b959-a1cf-456f-bef8-3405fe76ffec None None] MFA Rules not processed for user `7a7b54e21f3142dfae74495c8bcb6511`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:10.303386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2102b959-a1cf-456f-bef8-3405fe76ffec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:10.369167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1226/2471] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:10.380682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33169cf1-eb99-4015-bd0c-64f7a19bf98d admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:10.382499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-33169cf1-eb99-4015-bd0c-64f7a19bf98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:10.461835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33169cf1-eb99-4015-bd0c-64f7a19bf98d admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:10.466700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800d184-abc0-4075-b110-ca556ef070d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:10.467377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800d184-abc0-4075-b110-ca556ef070d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:10.467664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800d184-abc0-4075-b110-ca556ef070d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:10.468074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800d184-abc0-4075-b110-ca556ef070d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:10.478983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800d184-abc0-4075-b110-ca556ef070d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:10.480116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800d184-abc0-4075-b110-ca556ef070d9 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:10.492074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1246/2472] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:18.994152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102b959-a1cf-456f-bef8-3405fe76ffec None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:19.062551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102b959-a1cf-456f-bef8-3405fe76ffec None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:19.067028 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a860bc3-5548-42f6-9e2e-1d2f798b47a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:19.067668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a860bc3-5548-42f6-9e2e-1d2f798b47a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:19.067948 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a860bc3-5548-42f6-9e2e-1d2f798b47a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:19.068223 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a860bc3-5548-42f6-9e2e-1d2f798b47a5 admin admin] PATH_INFO: `/v3/users/7a7b54e21f3142dfae74495c8bcb6511` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:19.069368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a860bc3-5548-42f6-9e2e-1d2f798b47a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a7b54e21f3142dfae74495c8bcb6511)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:19.071288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a860bc3-5548-42f6-9e2e-1d2f798b47a5 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:19.137499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4a860bc3-5548-42f6-9e2e-1d2f798b47a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a7b54e21f3142dfae74495c8bcb6511'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:19.138678 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800d184-abc0-4075-b110-ca556ef070d9 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:19.145366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4a860bc3-5548-42f6-9e2e-1d2f798b47a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a7b54e21f3142dfae74495c8bcb6511'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:19.146045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4a860bc3-5548-42f6-9e2e-1d2f798b47a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a7b54e21f3142dfae74495c8bcb6511'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:19.146654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4a860bc3-5548-42f6-9e2e-1d2f798b47a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a7b54e21f3142dfae74495c8bcb6511'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:19.147901 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800d184-abc0-4075-b110-ca556ef070d9 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:19.149451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:19.151960 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7badb3ff-0b2c-4d64-b1ae-1a69dcbb9d9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:19.152713 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badb3ff-0b2c-4d64-b1ae-1a69dcbb9d9d service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:19.153032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badb3ff-0b2c-4d64-b1ae-1a69dcbb9d9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:19.153393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7badb3ff-0b2c-4d64-b1ae-1a69dcbb9d9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:19.155464 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7badb3ff-0b2c-4d64-b1ae-1a69dcbb9d9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:19.156088 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4a860bc3-5548-42f6-9e2e-1d2f798b47a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a7b54e21f3142dfae74495c8bcb6511'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:19.177354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1227/2473] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:49:18 2022] DELETE /identity/v3/users/7a7b54e21f3142dfae74495c8bcb6511 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:19.181932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50253771-846c-4833-bec7-66305a4db3d3 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:19.182223 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50253771-846c-4833-bec7-66305a4db3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:19.182502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50253771-846c-4833-bec7-66305a4db3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:19.202334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-50253771-846c-4833-bec7-66305a4db3d3 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:19.203569 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-50253771-846c-4833-bec7-66305a4db3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:19.243701 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7badb3ff-0b2c-4d64-b1ae-1a69dcbb9d9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:19.246126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7badb3ff-0b2c-4d64-b1ae-1a69dcbb9d9d service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:19.257341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:19.266673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1228/2474] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:19.267570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1247/2475] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:19.277252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50253771-846c-4833-bec7-66305a4db3d3 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:19.278585 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-50253771-846c-4833-bec7-66305a4db3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:19.338996 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50253771-846c-4833-bec7-66305a4db3d3 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:19.342877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d808838-3353-4f8f-9b73-ee2c5e632026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:19.343503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d808838-3353-4f8f-9b73-ee2c5e632026 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:19.343753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d808838-3353-4f8f-9b73-ee2c5e632026 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:19.344029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d808838-3353-4f8f-9b73-ee2c5e632026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:19.354370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d808838-3353-4f8f-9b73-ee2c5e632026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:19.356500 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d808838-3353-4f8f-9b73-ee2c5e632026 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:19.367700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1229/2476] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:19.666710 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7badb3ff-0b2c-4d64-b1ae-1a69dcbb9d9d service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:19.668284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7badb3ff-0b2c-4d64-b1ae-1a69dcbb9d9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:19.682920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7badb3ff-0b2c-4d64-b1ae-1a69dcbb9d9d service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:19.688249 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566cdfb-699c-4b64-8013-0eaf612ad2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:19.688896 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566cdfb-699c-4b64-8013-0eaf612ad2df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:19.689205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566cdfb-699c-4b64-8013-0eaf612ad2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:19.689479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566cdfb-699c-4b64-8013-0eaf612ad2df admin admin] PATH_INFO: `/v3/projects/a61f5e00c9434810994bca8eb23255dc` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:19.691121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566cdfb-699c-4b64-8013-0eaf612ad2df admin admin] RBAC: Authorizing `identity:delete_project(project_id=a61f5e00c9434810994bca8eb23255dc)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:19.693011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566cdfb-699c-4b64-8013-0eaf612ad2df admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:19.750637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8566cdfb-699c-4b64-8013-0eaf612ad2df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a61f5e00c9434810994bca8eb23255dc', 'request_id': 'req-8566cdfb-699c-4b64-8013-0eaf612ad2df'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:19.758380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8566cdfb-699c-4b64-8013-0eaf612ad2df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a61f5e00c9434810994bca8eb23255dc', 'request_id': 'req-8566cdfb-699c-4b64-8013-0eaf612ad2df'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:19.764810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8566cdfb-699c-4b64-8013-0eaf612ad2df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a61f5e00c9434810994bca8eb23255dc', 'request_id': 'req-8566cdfb-699c-4b64-8013-0eaf612ad2df'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:19.778208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8566cdfb-699c-4b64-8013-0eaf612ad2df admin admin] The token cache is being invalidate because project a61f5e00c9434810994bca8eb23255dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:49:19.778745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8566cdfb-699c-4b64-8013-0eaf612ad2df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:19.779374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8566cdfb-699c-4b64-8013-0eaf612ad2df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:19.780849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1248/2477] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:49:19 2022] DELETE /identity/v3/projects/a61f5e00c9434810994bca8eb23255dc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:19.786101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a704c11-bcbe-4857-83b8-a3403dbcfd46 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:19.786384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a704c11-bcbe-4857-83b8-a3403dbcfd46 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:19.786678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a704c11-bcbe-4857-83b8-a3403dbcfd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:19.806024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-7a704c11-bcbe-4857-83b8-a3403dbcfd46 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:19.807237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7a704c11-bcbe-4857-83b8-a3403dbcfd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:19.865294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1230/2478] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:19.873390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf9fc-26d1-483c-9a1e-a1506dc6ddf7 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:19.873676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf9fc-26d1-483c-9a1e-a1506dc6ddf7 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:19.874051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf9fc-26d1-483c-9a1e-a1506dc6ddf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:19.895013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b60cf9fc-26d1-483c-9a1e-a1506dc6ddf7 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:19.896216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b60cf9fc-26d1-483c-9a1e-a1506dc6ddf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:19.904084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1249/2479] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:19.908246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a704c11-bcbe-4857-83b8-a3403dbcfd46 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:19.917789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a704c11-bcbe-4857-83b8-a3403dbcfd46 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:19.922500 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5f6d8-9895-46d5-addd-0b26d211d4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:19.923084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5f6d8-9895-46d5-addd-0b26d211d4cc admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:19.923360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5f6d8-9895-46d5-addd-0b26d211d4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:19.923626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5f6d8-9895-46d5-addd-0b26d211d4cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:19.924261 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5f6d8-9895-46d5-addd-0b26d211d4cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:49:19.924560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5f6d8-9895-46d5-addd-0b26d211d4cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:19.926709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5f6d8-9895-46d5-addd-0b26d211d4cc admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:19.938431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1231/2480] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:49:19.943567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cf9fc-26d1-483c-9a1e-a1506dc6ddf7 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:19.953019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cf9fc-26d1-483c-9a1e-a1506dc6ddf7 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:19.959361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e074b-58f4-4f91-b5f8-84fe250298c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:19.960081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e074b-58f4-4f91-b5f8-84fe250298c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:19.960501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e074b-58f4-4f91-b5f8-84fe250298c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:19.960782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e074b-58f4-4f91-b5f8-84fe250298c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:19.961584 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e074b-58f4-4f91-b5f8-84fe250298c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:19.963479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e074b-58f4-4f91-b5f8-84fe250298c6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:19.992065 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1250/2481] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:49:19 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:19.996416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5f6d8-9895-46d5-addd-0b26d211d4cc admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:20.007120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5f6d8-9895-46d5-addd-0b26d211d4cc admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:20.010626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:20.076145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50b62e-ab37-4cde-9a2a-b576b0578e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:20.076765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50b62e-ab37-4cde-9a2a-b576b0578e3c admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:20.077045 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50b62e-ab37-4cde-9a2a-b576b0578e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:20.077351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50b62e-ab37-4cde-9a2a-b576b0578e3c admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:20.077999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50b62e-ab37-4cde-9a2a-b576b0578e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:20.079936 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50b62e-ab37-4cde-9a2a-b576b0578e3c admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:20.103063 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1232/2482] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:49:19 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:20.107458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e074b-58f4-4f91-b5f8-84fe250298c6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:20.118591 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e074b-58f4-4f91-b5f8-84fe250298c6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:20.123726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a69aa-6a07-49e7-8754-aff1ccc85cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:20.124343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a69aa-6a07-49e7-8754-aff1ccc85cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:20.124804 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a69aa-6a07-49e7-8754-aff1ccc85cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:20.125081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a69aa-6a07-49e7-8754-aff1ccc85cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:20.125827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a69aa-6a07-49e7-8754-aff1ccc85cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:20.127449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a69aa-6a07-49e7-8754-aff1ccc85cb9 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:20.139365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1251/2483] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:49:20.143354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50b62e-ab37-4cde-9a2a-b576b0578e3c admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:20.153831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50b62e-ab37-4cde-9a2a-b576b0578e3c admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:20.158809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2008b-f6fc-4fb5-a2a1-747c9fb87847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:20.159477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2008b-f6fc-4fb5-a2a1-747c9fb87847 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:20.159761 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2008b-f6fc-4fb5-a2a1-747c9fb87847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:20.160038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2008b-f6fc-4fb5-a2a1-747c9fb87847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:20.160687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2008b-f6fc-4fb5-a2a1-747c9fb87847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:20.162480 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2008b-f6fc-4fb5-a2a1-747c9fb87847 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:20.174403 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1233/2484] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:49:20.179280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a69aa-6a07-49e7-8754-aff1ccc85cb9 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:20.189974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a69aa-6a07-49e7-8754-aff1ccc85cb9 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:20.195657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d024b24-bd0c-4a68-9878-b7efd02d2263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:20.196440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d024b24-bd0c-4a68-9878-b7efd02d2263 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:20.196722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d024b24-bd0c-4a68-9878-b7efd02d2263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:20.197005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d024b24-bd0c-4a68-9878-b7efd02d2263 admin admin] PATH_INFO: `/v3/projects/5adf2ceefccd4080a904c554886d9577/users/cd2b4de9bda24160bcda78a9cdce2832/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:20.212961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d024b24-bd0c-4a68-9878-b7efd02d2263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5adf2ceefccd4080a904c554886d9577, user_id=cd2b4de9bda24160bcda78a9cdce2832, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:20.215688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d024b24-bd0c-4a68-9878-b7efd02d2263 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:20.229378 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1252/2485] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:49:20 2022] PUT /identity/v3/projects/5adf2ceefccd4080a904c554886d9577/users/cd2b4de9bda24160bcda78a9cdce2832/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:20.241655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9dee4-a5a2-46ae-a3f0-3b7095e07503 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:20.241932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9dee4-a5a2-46ae-a3f0-3b7095e07503 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:20.242240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9dee4-a5a2-46ae-a3f0-3b7095e07503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:20.272282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-77d9dee4-a5a2-46ae-a3f0-3b7095e07503 None None] MFA Rules not processed for user `cd2b4de9bda24160bcda78a9cdce2832`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:20.273535 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-77d9dee4-a5a2-46ae-a3f0-3b7095e07503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:20.340912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1234/2486] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:20.352319 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d024b24-bd0c-4a68-9878-b7efd02d2263 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:20.354028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4d024b24-bd0c-4a68-9878-b7efd02d2263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:20.414796 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d024b24-bd0c-4a68-9878-b7efd02d2263 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:20.418510 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62f2d1-62d7-4397-b60c-d68934a70dc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:20.419099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62f2d1-62d7-4397-b60c-d68934a70dc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:20.419372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62f2d1-62d7-4397-b60c-d68934a70dc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:20.419640 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62f2d1-62d7-4397-b60c-d68934a70dc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:20.429268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62f2d1-62d7-4397-b60c-d68934a70dc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:20.430662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62f2d1-62d7-4397-b60c-d68934a70dc1 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:20.440238 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1253/2487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:22.973411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9dee4-a5a2-46ae-a3f0-3b7095e07503 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:22.974818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-77d9dee4-a5a2-46ae-a3f0-3b7095e07503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:23.037531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9dee4-a5a2-46ae-a3f0-3b7095e07503 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:23.041110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352a83e-fcfd-45da-bc5c-ff41b511d5ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:23.041715 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352a83e-fcfd-45da-bc5c-ff41b511d5ed service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:23.041991 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352a83e-fcfd-45da-bc5c-ff41b511d5ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:23.042260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352a83e-fcfd-45da-bc5c-ff41b511d5ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:23.043975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8352a83e-fcfd-45da-bc5c-ff41b511d5ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:23.073484 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:23.108199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352a83e-fcfd-45da-bc5c-ff41b511d5ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:23.109696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352a83e-fcfd-45da-bc5c-ff41b511d5ed service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:23.119579 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1235/2488] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:26.285976 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62f2d1-62d7-4397-b60c-d68934a70dc1 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:26.287511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3b62f2d1-62d7-4397-b60c-d68934a70dc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:26.303176 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62f2d1-62d7-4397-b60c-d68934a70dc1 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:26.307925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fec382-d50d-4c95-bffd-d57f247968cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:26.308566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fec382-d50d-4c95-bffd-d57f247968cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:26.308846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fec382-d50d-4c95-bffd-d57f247968cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:26.309160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fec382-d50d-4c95-bffd-d57f247968cb admin admin] PATH_INFO: `/v3/users/7e3b9ceb95374f1dbf356122c8ed463c` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:26.310313 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fec382-d50d-4c95-bffd-d57f247968cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3b9ceb95374f1dbf356122c8ed463c)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:26.312227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fec382-d50d-4c95-bffd-d57f247968cb admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:26.326478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:26.368630 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f6fec382-d50d-4c95-bffd-d57f247968cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3b9ceb95374f1dbf356122c8ed463c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:26.382899 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f6fec382-d50d-4c95-bffd-d57f247968cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3b9ceb95374f1dbf356122c8ed463c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:26.383800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f6fec382-d50d-4c95-bffd-d57f247968cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3b9ceb95374f1dbf356122c8ed463c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:26.394341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f6fec382-d50d-4c95-bffd-d57f247968cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3b9ceb95374f1dbf356122c8ed463c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:26.395133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f6fec382-d50d-4c95-bffd-d57f247968cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3b9ceb95374f1dbf356122c8ed463c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:26.410085 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1254/2489] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:49:26 2022] DELETE /identity/v3/users/7e3b9ceb95374f1dbf356122c8ed463c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:26.598003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352a83e-fcfd-45da-bc5c-ff41b511d5ed service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:26.609734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352a83e-fcfd-45da-bc5c-ff41b511d5ed service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:26.669388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898e671-e515-471b-8322-473b61fdd613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:26.670044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898e671-e515-471b-8322-473b61fdd613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:26.670328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898e671-e515-471b-8322-473b61fdd613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:26.670604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898e671-e515-471b-8322-473b61fdd613 admin admin] PATH_INFO: `/v3/projects/6bc96052a69e409498462a1e4d499910` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:26.671975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898e671-e515-471b-8322-473b61fdd613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bc96052a69e409498462a1e4d499910)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:26.674025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898e671-e515-471b-8322-473b61fdd613 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:26.735032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f898e671-e515-471b-8322-473b61fdd613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bc96052a69e409498462a1e4d499910', 'request_id': 'req-f898e671-e515-471b-8322-473b61fdd613'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:26.741503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f898e671-e515-471b-8322-473b61fdd613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bc96052a69e409498462a1e4d499910', 'request_id': 'req-f898e671-e515-471b-8322-473b61fdd613'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:26.749269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f898e671-e515-471b-8322-473b61fdd613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bc96052a69e409498462a1e4d499910', 'request_id': 'req-f898e671-e515-471b-8322-473b61fdd613'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:26.762780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f898e671-e515-471b-8322-473b61fdd613 admin admin] The token cache is being invalidate because project 6bc96052a69e409498462a1e4d499910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:49:26.763071 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f898e671-e515-471b-8322-473b61fdd613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:26.763721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-f898e671-e515-471b-8322-473b61fdd613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:26.765238 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1236/2490] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:49:26 2022] DELETE /identity/v3/projects/6bc96052a69e409498462a1e4d499910 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:26.770374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56b792-2e7c-449a-bc9c-68d86098d730 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:26.770667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56b792-2e7c-449a-bc9c-68d86098d730 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:26.770977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56b792-2e7c-449a-bc9c-68d86098d730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:26.790345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-7d56b792-2e7c-449a-bc9c-68d86098d730 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:26.791560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7d56b792-2e7c-449a-bc9c-68d86098d730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:26.852064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1255/2491] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:26.859718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42223dd-8295-4b40-b979-886091ab1a06 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:26.859965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42223dd-8295-4b40-b979-886091ab1a06 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:26.860243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42223dd-8295-4b40-b979-886091ab1a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:26.882207 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-b42223dd-8295-4b40-b979-886091ab1a06 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:26.883422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b42223dd-8295-4b40-b979-886091ab1a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:26.891299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1237/2492] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:26.895503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56b792-2e7c-449a-bc9c-68d86098d730 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:26.905277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56b792-2e7c-449a-bc9c-68d86098d730 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:26.909916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7984da6-1d55-4f06-973d-895b4f234b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:26.910520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7984da6-1d55-4f06-973d-895b4f234b3d admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:26.910797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7984da6-1d55-4f06-973d-895b4f234b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:26.911066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7984da6-1d55-4f06-973d-895b4f234b3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:26.911735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7984da6-1d55-4f06-973d-895b4f234b3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:49:26.912033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7984da6-1d55-4f06-973d-895b4f234b3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:26.913637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7984da6-1d55-4f06-973d-895b4f234b3d admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:26.925932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1256/2493] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:49:26.930139 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42223dd-8295-4b40-b979-886091ab1a06 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:26.939289 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42223dd-8295-4b40-b979-886091ab1a06 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:26.944658 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448add60-f76e-494b-825a-1bed4e96e596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:26.945289 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448add60-f76e-494b-825a-1bed4e96e596 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:26.945569 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448add60-f76e-494b-825a-1bed4e96e596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:26.945840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448add60-f76e-494b-825a-1bed4e96e596 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:26.946473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448add60-f76e-494b-825a-1bed4e96e596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:26.948348 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448add60-f76e-494b-825a-1bed4e96e596 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:26.974787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1238/2494] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:49:26 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:26.978842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7984da6-1d55-4f06-973d-895b4f234b3d admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:26.989354 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7984da6-1d55-4f06-973d-895b4f234b3d admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:27.046585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc882b84-a0a2-4ca0-bdb1-b9583be2d9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:27.047205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc882b84-a0a2-4ca0-bdb1-b9583be2d9aa admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:27.047485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc882b84-a0a2-4ca0-bdb1-b9583be2d9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:27.047760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc882b84-a0a2-4ca0-bdb1-b9583be2d9aa admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:27.048393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc882b84-a0a2-4ca0-bdb1-b9583be2d9aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:27.050898 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc882b84-a0a2-4ca0-bdb1-b9583be2d9aa admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:27.072735 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1257/2495] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:49:26 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:27.076919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448add60-f76e-494b-825a-1bed4e96e596 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:27.087245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448add60-f76e-494b-825a-1bed4e96e596 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:27.092047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b92e5f-aee9-42b3-a699-8dc80ac035c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:27.092642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b92e5f-aee9-42b3-a699-8dc80ac035c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:27.092920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b92e5f-aee9-42b3-a699-8dc80ac035c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:27.093216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b92e5f-aee9-42b3-a699-8dc80ac035c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:27.093881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b92e5f-aee9-42b3-a699-8dc80ac035c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:27.095455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b92e5f-aee9-42b3-a699-8dc80ac035c7 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:27.109451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1239/2496] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:49:27.114255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc882b84-a0a2-4ca0-bdb1-b9583be2d9aa admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:27.124680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc882b84-a0a2-4ca0-bdb1-b9583be2d9aa admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:27.129693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363795d-8807-4432-bd4c-9bad0ef171b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:27.130299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363795d-8807-4432-bd4c-9bad0ef171b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:27.130585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363795d-8807-4432-bd4c-9bad0ef171b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:27.130886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363795d-8807-4432-bd4c-9bad0ef171b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:27.131645 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363795d-8807-4432-bd4c-9bad0ef171b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:27.133085 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363795d-8807-4432-bd4c-9bad0ef171b6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:27.135168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:27.146941 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1258/2497] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:49:27.151001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b92e5f-aee9-42b3-a699-8dc80ac035c7 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:27.162323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b92e5f-aee9-42b3-a699-8dc80ac035c7 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:27.167438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498dd5a6-53c2-44f1-b9d9-2a6323528e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:27.168108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498dd5a6-53c2-44f1-b9d9-2a6323528e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:27.168389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498dd5a6-53c2-44f1-b9d9-2a6323528e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:27.168664 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498dd5a6-53c2-44f1-b9d9-2a6323528e21 admin admin] PATH_INFO: `/v3/projects/6c7b3e2580f947b981190ee592fb5525/users/077aa647333e4e6e8350dec490cecb72/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:27.182456 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:27.186429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498dd5a6-53c2-44f1-b9d9-2a6323528e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c7b3e2580f947b981190ee592fb5525, user_id=077aa647333e4e6e8350dec490cecb72, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:27.189902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498dd5a6-53c2-44f1-b9d9-2a6323528e21 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:27.209009 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1240/2498] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:49:27 2022] PUT /identity/v3/projects/6c7b3e2580f947b981190ee592fb5525/users/077aa647333e4e6e8350dec490cecb72/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:49:27.225743 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4965bf8-0485-40db-b817-6ee6490fbdc0 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:27.226145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4965bf8-0485-40db-b817-6ee6490fbdc0 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:27.226570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4965bf8-0485-40db-b817-6ee6490fbdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:27.258045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e4965bf8-0485-40db-b817-6ee6490fbdc0 None None] MFA Rules not processed for user `077aa647333e4e6e8350dec490cecb72`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:27.259265 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e4965bf8-0485-40db-b817-6ee6490fbdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:27.341466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1259/2499] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:27.353928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498dd5a6-53c2-44f1-b9d9-2a6323528e21 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:27.355706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-498dd5a6-53c2-44f1-b9d9-2a6323528e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:27.434166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498dd5a6-53c2-44f1-b9d9-2a6323528e21 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:27.443879 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd41ce4-e342-4bfb-a82f-b1a6c1cb7c25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:27.444497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd41ce4-e342-4bfb-a82f-b1a6c1cb7c25 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:27.444761 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd41ce4-e342-4bfb-a82f-b1a6c1cb7c25 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:27.445032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd41ce4-e342-4bfb-a82f-b1a6c1cb7c25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:27.454038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd41ce4-e342-4bfb-a82f-b1a6c1cb7c25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:27.455725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd41ce4-e342-4bfb-a82f-b1a6c1cb7c25 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:27.465842 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1241/2500] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:30.005661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4965bf8-0485-40db-b817-6ee6490fbdc0 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:30.007113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e4965bf8-0485-40db-b817-6ee6490fbdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:30.070905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4965bf8-0485-40db-b817-6ee6490fbdc0 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:30.075423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:30.076046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:30.076324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:30.076594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] PATH_INFO: `/v3/users/cd2b4de9bda24160bcda78a9cdce2832` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:30.077729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd2b4de9bda24160bcda78a9cdce2832)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:30.079634 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:30.129718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd2b4de9bda24160bcda78a9cdce2832'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:30.141776 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd2b4de9bda24160bcda78a9cdce2832'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:30.142453 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd2b4de9bda24160bcda78a9cdce2832'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:30.149983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd2b4de9bda24160bcda78a9cdce2832'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:30.150612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd2b4de9bda24160bcda78a9cdce2832'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:30.169175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1260/2501] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:49:30 2022] DELETE /identity/v3/users/cd2b4de9bda24160bcda78a9cdce2832 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:30.173998 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a60bf7-15cb-4c2a-aab4-898c71de4d8b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:30.174499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a60bf7-15cb-4c2a-aab4-898c71de4d8b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:30.174634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a60bf7-15cb-4c2a-aab4-898c71de4d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:30.194359 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-94a60bf7-15cb-4c2a-aab4-898c71de4d8b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:30.195543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-94a60bf7-15cb-4c2a-aab4-898c71de4d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:30.258855 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1242/2502] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:30.269012 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:30.270500 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:30.277229 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:30.332973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3412ff7-78e9-4854-97bf-a83960998ff1 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:30.336606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b77a41-8c5a-4f3c-b7ea-8e22d88ea298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:30.337246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b77a41-8c5a-4f3c-b7ea-8e22d88ea298 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:30.337527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b77a41-8c5a-4f3c-b7ea-8e22d88ea298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:30.337802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b77a41-8c5a-4f3c-b7ea-8e22d88ea298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:30.346795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b77a41-8c5a-4f3c-b7ea-8e22d88ea298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:30.348387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b77a41-8c5a-4f3c-b7ea-8e22d88ea298 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:30.358717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1261/2503] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:30.629203 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a60bf7-15cb-4c2a-aab4-898c71de4d8b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:30.630770 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-94a60bf7-15cb-4c2a-aab4-898c71de4d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:30.644175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a60bf7-15cb-4c2a-aab4-898c71de4d8b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:30.649866 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee730a-b1ba-4804-b63e-7500d7809959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:30.650668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee730a-b1ba-4804-b63e-7500d7809959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:30.650953 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee730a-b1ba-4804-b63e-7500d7809959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:30.651276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee730a-b1ba-4804-b63e-7500d7809959 admin admin] PATH_INFO: `/v3/projects/5adf2ceefccd4080a904c554886d9577` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:30.652447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee730a-b1ba-4804-b63e-7500d7809959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5adf2ceefccd4080a904c554886d9577)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:30.654346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee730a-b1ba-4804-b63e-7500d7809959 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:30.702757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bbee730a-b1ba-4804-b63e-7500d7809959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5adf2ceefccd4080a904c554886d9577', 'request_id': 'req-bbee730a-b1ba-4804-b63e-7500d7809959'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:30.708871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bbee730a-b1ba-4804-b63e-7500d7809959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5adf2ceefccd4080a904c554886d9577', 'request_id': 'req-bbee730a-b1ba-4804-b63e-7500d7809959'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:30.716388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bbee730a-b1ba-4804-b63e-7500d7809959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5adf2ceefccd4080a904c554886d9577', 'request_id': 'req-bbee730a-b1ba-4804-b63e-7500d7809959'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:30.726052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:30.730267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bbee730a-b1ba-4804-b63e-7500d7809959 admin admin] The token cache is being invalidate because project 5adf2ceefccd4080a904c554886d9577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:49:30.730895 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bbee730a-b1ba-4804-b63e-7500d7809959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:30.731862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-bbee730a-b1ba-4804-b63e-7500d7809959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:30.733608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1243/2504] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:49:30 2022] DELETE /identity/v3/projects/5adf2ceefccd4080a904c554886d9577 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:30.738839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff451617-7f95-4ce7-867b-0afeda4997f6 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:30.739126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff451617-7f95-4ce7-867b-0afeda4997f6 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:30.739428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff451617-7f95-4ce7-867b-0afeda4997f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:30.759667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-ff451617-7f95-4ce7-867b-0afeda4997f6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:30.760807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ff451617-7f95-4ce7-867b-0afeda4997f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:30.819915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1262/2505] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:30.827782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66474bbb-a4a6-42a4-af75-20209cef5b11 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:30.828114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66474bbb-a4a6-42a4-af75-20209cef5b11 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:30.828389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66474bbb-a4a6-42a4-af75-20209cef5b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:30.848805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-66474bbb-a4a6-42a4-af75-20209cef5b11 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:30.850036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-66474bbb-a4a6-42a4-af75-20209cef5b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:30.857809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1244/2506] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:30.862158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff451617-7f95-4ce7-867b-0afeda4997f6 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:30.872102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff451617-7f95-4ce7-867b-0afeda4997f6 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:30.876703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbff6b0-1cc1-4cd9-9cc4-a90b6fe56022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:30.877370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbff6b0-1cc1-4cd9-9cc4-a90b6fe56022 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:30.877652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbff6b0-1cc1-4cd9-9cc4-a90b6fe56022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:30.877925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbff6b0-1cc1-4cd9-9cc4-a90b6fe56022 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:30.878566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbff6b0-1cc1-4cd9-9cc4-a90b6fe56022 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:49:30.878870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbff6b0-1cc1-4cd9-9cc4-a90b6fe56022 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:30.880946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbff6b0-1cc1-4cd9-9cc4-a90b6fe56022 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:30.892561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1263/2507] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:49:30.897411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66474bbb-a4a6-42a4-af75-20209cef5b11 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:30.906555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66474bbb-a4a6-42a4-af75-20209cef5b11 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:30.911437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e91170-2b89-4bad-8509-925fe2ae6201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:30.912018 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e91170-2b89-4bad-8509-925fe2ae6201 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:30.912292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e91170-2b89-4bad-8509-925fe2ae6201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:30.912559 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e91170-2b89-4bad-8509-925fe2ae6201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:30.913224 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e91170-2b89-4bad-8509-925fe2ae6201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:30.915105 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e91170-2b89-4bad-8509-925fe2ae6201 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:30.943497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1245/2508] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:49:30 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:30.947775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbff6b0-1cc1-4cd9-9cc4-a90b6fe56022 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:30.959395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbff6b0-1cc1-4cd9-9cc4-a90b6fe56022 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:31.015115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f054933-53b9-4793-b4db-cf19f2612343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:31.015744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f054933-53b9-4793-b4db-cf19f2612343 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:31.015890 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f054933-53b9-4793-b4db-cf19f2612343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:31.016166 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f054933-53b9-4793-b4db-cf19f2612343 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:31.016813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f054933-53b9-4793-b4db-cf19f2612343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:31.018761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f054933-53b9-4793-b4db-cf19f2612343 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:31.041468 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1264/2509] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:49:30 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:49:31.045764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e91170-2b89-4bad-8509-925fe2ae6201 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:31.056352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e91170-2b89-4bad-8509-925fe2ae6201 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:31.061252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae593c-417c-44e6-8c3b-35ee4a227692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:31.061848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae593c-417c-44e6-8c3b-35ee4a227692 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:31.062123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae593c-417c-44e6-8c3b-35ee4a227692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:31.062393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae593c-417c-44e6-8c3b-35ee4a227692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:31.063057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae593c-417c-44e6-8c3b-35ee4a227692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:31.064669 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae593c-417c-44e6-8c3b-35ee4a227692 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:31.076046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1246/2510] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:49:31.079963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f054933-53b9-4793-b4db-cf19f2612343 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:31.090628 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f054933-53b9-4793-b4db-cf19f2612343 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:31.095713 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06ba8f-4e4e-4f81-af70-b9b097a2c19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:31.096304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06ba8f-4e4e-4f81-af70-b9b097a2c19b admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:31.096583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06ba8f-4e4e-4f81-af70-b9b097a2c19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:31.096860 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06ba8f-4e4e-4f81-af70-b9b097a2c19b admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:31.097546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06ba8f-4e4e-4f81-af70-b9b097a2c19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:31.099112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06ba8f-4e4e-4f81-af70-b9b097a2c19b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:31.109774 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1265/2511] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:49:31.114239 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae593c-417c-44e6-8c3b-35ee4a227692 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:31.124185 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae593c-417c-44e6-8c3b-35ee4a227692 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:31.128744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9809a9-b5c8-4fd5-8a54-5684cee94d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:31.130047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9809a9-b5c8-4fd5-8a54-5684cee94d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:31.130329 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9809a9-b5c8-4fd5-8a54-5684cee94d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:31.130605 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9809a9-b5c8-4fd5-8a54-5684cee94d06 admin admin] PATH_INFO: `/v3/projects/b6df65c33dc3480c91d62e90af814f22/users/6ddad0e90dba4b258a8c3cfdf128b9f6/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:31.145066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9809a9-b5c8-4fd5-8a54-5684cee94d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6df65c33dc3480c91d62e90af814f22, user_id=6ddad0e90dba4b258a8c3cfdf128b9f6, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:31.147777 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9809a9-b5c8-4fd5-8a54-5684cee94d06 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:31.161011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1247/2512] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:49:31 2022] PUT /identity/v3/projects/b6df65c33dc3480c91d62e90af814f22/users/6ddad0e90dba4b258a8c3cfdf128b9f6/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:31.172970 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a6eb-2768-4429-b836-a3191cf3b4cd None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:31.173436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a6eb-2768-4429-b836-a3191cf3b4cd None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:31.173813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2a6eb-2768-4429-b836-a3191cf3b4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:31.200303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:31.204268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-64f2a6eb-2768-4429-b836-a3191cf3b4cd None None] MFA Rules not processed for user `6ddad0e90dba4b258a8c3cfdf128b9f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:31.205980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-64f2a6eb-2768-4429-b836-a3191cf3b4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:31.273754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1266/2513] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:31.284750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9809a9-b5c8-4fd5-8a54-5684cee94d06 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:31.286598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7f9809a9-b5c8-4fd5-8a54-5684cee94d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:31.348195 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9809a9-b5c8-4fd5-8a54-5684cee94d06 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:31.351848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b241c3-7630-43da-a8fc-b6c6aea66a63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:31.352442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b241c3-7630-43da-a8fc-b6c6aea66a63 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:31.352719 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b241c3-7630-43da-a8fc-b6c6aea66a63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:31.352987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b241c3-7630-43da-a8fc-b6c6aea66a63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:31.361208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b241c3-7630-43da-a8fc-b6c6aea66a63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:31.362625 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b241c3-7630-43da-a8fc-b6c6aea66a63 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:31.372327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1248/2514] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:39.043356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2a6eb-2768-4429-b836-a3191cf3b4cd None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:39.043356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-64f2a6eb-2768-4429-b836-a3191cf3b4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:39.106797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2a6eb-2768-4429-b836-a3191cf3b4cd None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:39.111241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:39.111862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:39.112140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:39.112412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] PATH_INFO: `/v3/users/077aa647333e4e6e8350dec490cecb72` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:39.113563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] RBAC: Authorizing `identity:delete_user(user_id=077aa647333e4e6e8350dec490cecb72)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:39.116102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:39.162697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '077aa647333e4e6e8350dec490cecb72'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:39.174868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '077aa647333e4e6e8350dec490cecb72'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:39.175467 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '077aa647333e4e6e8350dec490cecb72'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:39.183039 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '077aa647333e4e6e8350dec490cecb72'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:39.183693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '077aa647333e4e6e8350dec490cecb72'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:39.196392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1267/2515] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:49:39 2022] DELETE /identity/v3/users/077aa647333e4e6e8350dec490cecb72 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:39.201081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e05b12-679b-49ed-a361-36992d1c7290 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:39.201605 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e05b12-679b-49ed-a361-36992d1c7290 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:39.201711 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e05b12-679b-49ed-a361-36992d1c7290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:39.220287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-78e05b12-679b-49ed-a361-36992d1c7290 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:39.221518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-78e05b12-679b-49ed-a361-36992d1c7290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:39.283338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1249/2516] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:39.293727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:39.295168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:39.322482 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:39.355787 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288ef22-c456-4f50-8bc2-923d34c1a80f admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:39.359519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68919fd8-bc2a-485b-bce0-9092ad274cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:39.360126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68919fd8-bc2a-485b-bce0-9092ad274cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:39.360409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68919fd8-bc2a-485b-bce0-9092ad274cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:39.360684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68919fd8-bc2a-485b-bce0-9092ad274cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:39.369750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68919fd8-bc2a-485b-bce0-9092ad274cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:39.371197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68919fd8-bc2a-485b-bce0-9092ad274cb4 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:39.380641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1268/2517] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:39.654726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e05b12-679b-49ed-a361-36992d1c7290 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:39.656186 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-78e05b12-679b-49ed-a361-36992d1c7290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:39.668650 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e05b12-679b-49ed-a361-36992d1c7290 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:39.673603 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9264f8-abee-496c-a16a-de963b9ca5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:39.674231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9264f8-abee-496c-a16a-de963b9ca5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:39.674505 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9264f8-abee-496c-a16a-de963b9ca5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:39.674774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9264f8-abee-496c-a16a-de963b9ca5b3 admin admin] PATH_INFO: `/v3/projects/6c7b3e2580f947b981190ee592fb5525` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:39.675880 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9264f8-abee-496c-a16a-de963b9ca5b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c7b3e2580f947b981190ee592fb5525)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:39.677815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9264f8-abee-496c-a16a-de963b9ca5b3 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:39.716634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:39.730847 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1d9264f8-abee-496c-a16a-de963b9ca5b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c7b3e2580f947b981190ee592fb5525', 'request_id': 'req-1d9264f8-abee-496c-a16a-de963b9ca5b3'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:39.740159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1d9264f8-abee-496c-a16a-de963b9ca5b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c7b3e2580f947b981190ee592fb5525', 'request_id': 'req-1d9264f8-abee-496c-a16a-de963b9ca5b3'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:39.749517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1d9264f8-abee-496c-a16a-de963b9ca5b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c7b3e2580f947b981190ee592fb5525', 'request_id': 'req-1d9264f8-abee-496c-a16a-de963b9ca5b3'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:39.761574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1d9264f8-abee-496c-a16a-de963b9ca5b3 admin admin] The token cache is being invalidate because project 6c7b3e2580f947b981190ee592fb5525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:49:39.761862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1d9264f8-abee-496c-a16a-de963b9ca5b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:39.762488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1d9264f8-abee-496c-a16a-de963b9ca5b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:39.763975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1250/2518] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:49:39 2022] DELETE /identity/v3/projects/6c7b3e2580f947b981190ee592fb5525 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:39.768972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647022b-85ff-48a9-8fe2-1a7cb6c0fcc4 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:39.769312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647022b-85ff-48a9-8fe2-1a7cb6c0fcc4 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:39.769590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647022b-85ff-48a9-8fe2-1a7cb6c0fcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:39.790326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b647022b-85ff-48a9-8fe2-1a7cb6c0fcc4 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:39.791576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b647022b-85ff-48a9-8fe2-1a7cb6c0fcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:39.850363 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1269/2519] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:39.858046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e24ee-90a2-4ebb-82d8-870f2f201b06 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:39.858328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e24ee-90a2-4ebb-82d8-870f2f201b06 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:39.858605 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e24ee-90a2-4ebb-82d8-870f2f201b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:39.878195 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-515e24ee-90a2-4ebb-82d8-870f2f201b06 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:39.879402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-515e24ee-90a2-4ebb-82d8-870f2f201b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:39.887740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1251/2520] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:39.891886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647022b-85ff-48a9-8fe2-1a7cb6c0fcc4 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:39.902128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647022b-85ff-48a9-8fe2-1a7cb6c0fcc4 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:39.906629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89403e-6814-46a1-8da3-60acd176ab55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:39.907217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89403e-6814-46a1-8da3-60acd176ab55 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:39.907491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89403e-6814-46a1-8da3-60acd176ab55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:39.907759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89403e-6814-46a1-8da3-60acd176ab55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:39.908398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89403e-6814-46a1-8da3-60acd176ab55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:49:39.908698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89403e-6814-46a1-8da3-60acd176ab55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:39.910299 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89403e-6814-46a1-8da3-60acd176ab55 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:39.922218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1270/2521] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:49:39.926634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e24ee-90a2-4ebb-82d8-870f2f201b06 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:39.935801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e24ee-90a2-4ebb-82d8-870f2f201b06 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:39.940611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5f8d9-8d9f-4364-abaa-6b622447106d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:39.941252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5f8d9-8d9f-4364-abaa-6b622447106d admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:39.941537 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5f8d9-8d9f-4364-abaa-6b622447106d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:39.941808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5f8d9-8d9f-4364-abaa-6b622447106d admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:39.942926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5f8d9-8d9f-4364-abaa-6b622447106d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:39.944818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5f8d9-8d9f-4364-abaa-6b622447106d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:39.972199 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1252/2522] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:49:39 2022] POST /identity/v3/projects => generated 357 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:39.976626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89403e-6814-46a1-8da3-60acd176ab55 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:39.986523 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89403e-6814-46a1-8da3-60acd176ab55 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:40.042769 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49075a-7487-4c6a-9423-e5768dcec772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:40.043365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49075a-7487-4c6a-9423-e5768dcec772 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.043936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49075a-7487-4c6a-9423-e5768dcec772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.043936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49075a-7487-4c6a-9423-e5768dcec772 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.044499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49075a-7487-4c6a-9423-e5768dcec772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:40.046668 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49075a-7487-4c6a-9423-e5768dcec772 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:40.067163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1271/2523] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:49:39 2022] POST /identity/v3/users => generated 279 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:40.072006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5f8d9-8d9f-4364-abaa-6b622447106d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:40.082913 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5f8d9-8d9f-4364-abaa-6b622447106d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:40.091217 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e13e99-5e56-4cd8-a7c8-2cc8750bfbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:40.091847 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e13e99-5e56-4cd8-a7c8-2cc8750bfbb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.092128 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e13e99-5e56-4cd8-a7c8-2cc8750bfbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.092399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e13e99-5e56-4cd8-a7c8-2cc8750bfbb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.093058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e13e99-5e56-4cd8-a7c8-2cc8750bfbb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:40.094674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e13e99-5e56-4cd8-a7c8-2cc8750bfbb0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:40.105724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1253/2524] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:49:40.109884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49075a-7487-4c6a-9423-e5768dcec772 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:40.120417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49075a-7487-4c6a-9423-e5768dcec772 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:40.125027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606c8ff-07f0-432e-ad81-5db33de80ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:40.125646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606c8ff-07f0-432e-ad81-5db33de80ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.125925 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606c8ff-07f0-432e-ad81-5db33de80ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.126197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606c8ff-07f0-432e-ad81-5db33de80ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.126831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606c8ff-07f0-432e-ad81-5db33de80ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:40.128406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606c8ff-07f0-432e-ad81-5db33de80ad9 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:40.140047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1272/2525] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:49:40.144344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e13e99-5e56-4cd8-a7c8-2cc8750bfbb0 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:40.154546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e13e99-5e56-4cd8-a7c8-2cc8750bfbb0 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:40.159323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab9bb60-c92c-4289-9dac-6fadc0b2bf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:40.159972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9bb60-c92c-4289-9dac-6fadc0b2bf90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.160252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9bb60-c92c-4289-9dac-6fadc0b2bf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.160527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9bb60-c92c-4289-9dac-6fadc0b2bf90 admin admin] PATH_INFO: `/v3/projects/f471fbe510124f489638c4bc3a02c35d/users/7e1670c075894ec0b0c66797124912ce/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.175545 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab9bb60-c92c-4289-9dac-6fadc0b2bf90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f471fbe510124f489638c4bc3a02c35d, user_id=7e1670c075894ec0b0c66797124912ce, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:40.178753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab9bb60-c92c-4289-9dac-6fadc0b2bf90 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:40.192424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1254/2526] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:49:40 2022] PUT /identity/v3/projects/f471fbe510124f489638c4bc3a02c35d/users/7e1670c075894ec0b0c66797124912ce/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:40.203609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c22ec-99ca-4410-a5f2-3d0ae0cc6f0b None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.203903 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c22ec-99ca-4410-a5f2-3d0ae0cc6f0b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.204183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c22ec-99ca-4410-a5f2-3d0ae0cc6f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.232055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-421c22ec-99ca-4410-a5f2-3d0ae0cc6f0b None None] MFA Rules not processed for user `7e1670c075894ec0b0c66797124912ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:40.233215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-421c22ec-99ca-4410-a5f2-3d0ae0cc6f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:40.252050 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:40.299626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1273/2527] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:40.304608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab9bb60-c92c-4289-9dac-6fadc0b2bf90 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:40.366553 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab9bb60-c92c-4289-9dac-6fadc0b2bf90 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:40.371342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577e36e-2cf2-4b2c-8721-1a98dc12c73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:40.371943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577e36e-2cf2-4b2c-8721-1a98dc12c73f admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.372225 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577e36e-2cf2-4b2c-8721-1a98dc12c73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.372500 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577e36e-2cf2-4b2c-8721-1a98dc12c73f admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.373166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577e36e-2cf2-4b2c-8721-1a98dc12c73f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:40.375076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577e36e-2cf2-4b2c-8721-1a98dc12c73f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:40.402269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1255/2528] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:49:40 2022] POST /identity/v3/projects => generated 357 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:40.406516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c22ec-99ca-4410-a5f2-3d0ae0cc6f0b None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:40.417689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c22ec-99ca-4410-a5f2-3d0ae0cc6f0b None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:40.480240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17308c-d3d0-48ea-885f-40e921750108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:40.480850 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17308c-d3d0-48ea-885f-40e921750108 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.481162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17308c-d3d0-48ea-885f-40e921750108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.481440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17308c-d3d0-48ea-885f-40e921750108 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.482123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17308c-d3d0-48ea-885f-40e921750108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:40.483967 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17308c-d3d0-48ea-885f-40e921750108 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:40.506537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1274/2529] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:49:40 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:49:40.510816 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577e36e-2cf2-4b2c-8721-1a98dc12c73f admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:40.521445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577e36e-2cf2-4b2c-8721-1a98dc12c73f admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:40.529197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c5671-d408-4883-9923-cf15394866a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:40.529827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c5671-d408-4883-9923-cf15394866a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.530107 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c5671-d408-4883-9923-cf15394866a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.530383 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c5671-d408-4883-9923-cf15394866a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.531033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c5671-d408-4883-9923-cf15394866a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:40.532639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c5671-d408-4883-9923-cf15394866a4 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:40.544237 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1256/2530] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:49:40.548788 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17308c-d3d0-48ea-885f-40e921750108 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:40.560179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17308c-d3d0-48ea-885f-40e921750108 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:40.564865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e6be9-0cd0-4992-aa7f-fcfafb4b446c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:40.565551 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e6be9-0cd0-4992-aa7f-fcfafb4b446c admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.565835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e6be9-0cd0-4992-aa7f-fcfafb4b446c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.566114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e6be9-0cd0-4992-aa7f-fcfafb4b446c admin admin] PATH_INFO: `/v3/projects/15e2fa0fa67d48e6bee08ca45a071cec/users/5d4f7a5fb99048bf86b5b231dd1b7c9f/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.580998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e6be9-0cd0-4992-aa7f-fcfafb4b446c admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e2fa0fa67d48e6bee08ca45a071cec, user_id=5d4f7a5fb99048bf86b5b231dd1b7c9f, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:40.583696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e6be9-0cd0-4992-aa7f-fcfafb4b446c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:40.597113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1275/2531] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:49:40 2022] PUT /identity/v3/projects/15e2fa0fa67d48e6bee08ca45a071cec/users/5d4f7a5fb99048bf86b5b231dd1b7c9f/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:40.601610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c5671-d408-4883-9923-cf15394866a4 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:40.611675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c5671-d408-4883-9923-cf15394866a4 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:40.641570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:40.669717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c00d3-ac7c-4f9a-ac7f-a19cf5e037a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:40.670319 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c00d3-ac7c-4f9a-ac7f-a19cf5e037a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.670604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c00d3-ac7c-4f9a-ac7f-a19cf5e037a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.671141 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c00d3-ac7c-4f9a-ac7f-a19cf5e037a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.671780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c00d3-ac7c-4f9a-ac7f-a19cf5e037a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:40.673390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c00d3-ac7c-4f9a-ac7f-a19cf5e037a0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:40.684550 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1257/2532] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:49:40.688800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e6be9-0cd0-4992-aa7f-fcfafb4b446c admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:40.699136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e6be9-0cd0-4992-aa7f-fcfafb4b446c admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:40.704207 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2ad13-c135-4312-b5e0-84c9bbcf5544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:40.704853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2ad13-c135-4312-b5e0-84c9bbcf5544 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.705164 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2ad13-c135-4312-b5e0-84c9bbcf5544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.705445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2ad13-c135-4312-b5e0-84c9bbcf5544 admin admin] PATH_INFO: `/v3/projects/15e2fa0fa67d48e6bee08ca45a071cec/users/5d4f7a5fb99048bf86b5b231dd1b7c9f/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.707375 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2ad13-c135-4312-b5e0-84c9bbcf5544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e2fa0fa67d48e6bee08ca45a071cec, user_id=5d4f7a5fb99048bf86b5b231dd1b7c9f, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:40.710054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2ad13-c135-4312-b5e0-84c9bbcf5544 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:40.720602 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1276/2533] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:49:40 2022] PUT /identity/v3/projects/15e2fa0fa67d48e6bee08ca45a071cec/users/5d4f7a5fb99048bf86b5b231dd1b7c9f/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:40.734391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e10928-af33-40b0-b22c-184f2ecf66e0 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.734898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e10928-af33-40b0-b22c-184f2ecf66e0 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.735309 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e10928-af33-40b0-b22c-184f2ecf66e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.770111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-a7e10928-af33-40b0-b22c-184f2ecf66e0 None None] MFA Rules not processed for user `5d4f7a5fb99048bf86b5b231dd1b7c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:40.771337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a7e10928-af33-40b0-b22c-184f2ecf66e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:40.839849 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1258/2534] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:40 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:40.851814 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2ad13-c135-4312-b5e0-84c9bbcf5544 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:40.853637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a0b2ad13-c135-4312-b5e0-84c9bbcf5544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:40.909421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2ad13-c135-4312-b5e0-84c9bbcf5544 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:40.913633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8f2ab-684b-4153-ae4a-c0c2775cc341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:40.914227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8f2ab-684b-4153-ae4a-c0c2775cc341 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:40.914501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8f2ab-684b-4153-ae4a-c0c2775cc341 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:40.914767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8f2ab-684b-4153-ae4a-c0c2775cc341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:40.923924 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8f2ab-684b-4153-ae4a-c0c2775cc341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:40.925352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8f2ab-684b-4153-ae4a-c0c2775cc341 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:40.934663 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1277/2535] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:40 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:41.009459 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e10928-af33-40b0-b22c-184f2ecf66e0 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:41.019001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e10928-af33-40b0-b22c-184f2ecf66e0 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:41.023298 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d4ab7-32f6-4400-b034-4035c51f5c2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:41.023912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d4ab7-32f6-4400-b034-4035c51f5c2d service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:41.024175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d4ab7-32f6-4400-b034-4035c51f5c2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:41.024449 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d4ab7-32f6-4400-b034-4035c51f5c2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:41.081107 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d4ab7-32f6-4400-b034-4035c51f5c2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:41.083304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d4ab7-32f6-4400-b034-4035c51f5c2d service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:41.092725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1259/2536] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:41 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:50.930652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:51.088265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=76 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:56.421826 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8f2ab-684b-4153-ae4a-c0c2775cc341 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:56.423264 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-53e8f2ab-684b-4153-ae4a-c0c2775cc341 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:56.492633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8f2ab-684b-4153-ae4a-c0c2775cc341 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:56.499019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:56.499666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:56.499944 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:56.500218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] PATH_INFO: `/v3/users/6ddad0e90dba4b258a8c3cfdf128b9f6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:56.501386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ddad0e90dba4b258a8c3cfdf128b9f6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:56.503283 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:56.556786 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ddad0e90dba4b258a8c3cfdf128b9f6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:56.572737 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ddad0e90dba4b258a8c3cfdf128b9f6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:56.573547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ddad0e90dba4b258a8c3cfdf128b9f6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:56.581619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ddad0e90dba4b258a8c3cfdf128b9f6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:56.582605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ddad0e90dba4b258a8c3cfdf128b9f6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:56.597986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1278/2537] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:49:56 2022] DELETE /identity/v3/users/6ddad0e90dba4b258a8c3cfdf128b9f6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:56.602398 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf7dc5-3177-4e33-8c55-82ed6735cb5c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:56.602880 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf7dc5-3177-4e33-8c55-82ed6735cb5c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:56.602993 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf7dc5-3177-4e33-8c55-82ed6735cb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:56.623101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-2dbf7dc5-3177-4e33-8c55-82ed6735cb5c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:56.624322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2dbf7dc5-3177-4e33-8c55-82ed6735cb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:56.690521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1260/2538] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:56.701907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:56.703972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:56.768132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa13ca-6d68-4cfe-87c4-5740909c01f8 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:56.772547 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5511f-9a77-4294-b64b-82f88d8b4ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:56.773271 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5511f-9a77-4294-b64b-82f88d8b4ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:56.773557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5511f-9a77-4294-b64b-82f88d8b4ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:56.773909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5511f-9a77-4294-b64b-82f88d8b4ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:56.784700 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5511f-9a77-4294-b64b-82f88d8b4ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:56.786372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5511f-9a77-4294-b64b-82f88d8b4ea3 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:56.796604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1279/2539] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:57.078813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf7dc5-3177-4e33-8c55-82ed6735cb5c None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:57.080489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-2dbf7dc5-3177-4e33-8c55-82ed6735cb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:57.092649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf7dc5-3177-4e33-8c55-82ed6735cb5c None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:57.097568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228519a-96c4-4c09-9184-9ead24bf5c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:57.099458 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228519a-96c4-4c09-9184-9ead24bf5c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:57.099738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228519a-96c4-4c09-9184-9ead24bf5c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:57.100197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228519a-96c4-4c09-9184-9ead24bf5c76 admin admin] PATH_INFO: `/v3/projects/b6df65c33dc3480c91d62e90af814f22` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:57.103009 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228519a-96c4-4c09-9184-9ead24bf5c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6df65c33dc3480c91d62e90af814f22)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:57.105531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228519a-96c4-4c09-9184-9ead24bf5c76 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:57.159062 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2228519a-96c4-4c09-9184-9ead24bf5c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6df65c33dc3480c91d62e90af814f22', 'request_id': 'req-2228519a-96c4-4c09-9184-9ead24bf5c76'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:57.165405 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2228519a-96c4-4c09-9184-9ead24bf5c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6df65c33dc3480c91d62e90af814f22', 'request_id': 'req-2228519a-96c4-4c09-9184-9ead24bf5c76'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:57.173622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2228519a-96c4-4c09-9184-9ead24bf5c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6df65c33dc3480c91d62e90af814f22', 'request_id': 'req-2228519a-96c4-4c09-9184-9ead24bf5c76'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:57.187153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2228519a-96c4-4c09-9184-9ead24bf5c76 admin admin] The token cache is being invalidate because project b6df65c33dc3480c91d62e90af814f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:49:57.187447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2228519a-96c4-4c09-9184-9ead24bf5c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:57.188091 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2228519a-96c4-4c09-9184-9ead24bf5c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:57.189610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1261/2540] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:49:57 2022] DELETE /identity/v3/projects/b6df65c33dc3480c91d62e90af814f22 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:57.194685 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c191c-fe63-405e-bacf-39ae15e17aea None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:57.194974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c191c-fe63-405e-bacf-39ae15e17aea None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:57.195246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c191c-fe63-405e-bacf-39ae15e17aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:57.218992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-bd4c191c-fe63-405e-bacf-39ae15e17aea None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:57.220029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-bd4c191c-fe63-405e-bacf-39ae15e17aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:57.242486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:57.292077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1280/2541] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:57.300983 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94ea84-eb48-4c02-9da2-7da4520f9cef None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:57.301993 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94ea84-eb48-4c02-9da2-7da4520f9cef None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:57.302270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94ea84-eb48-4c02-9da2-7da4520f9cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:57.323411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-eb94ea84-eb48-4c02-9da2-7da4520f9cef None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:57.324626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-eb94ea84-eb48-4c02-9da2-7da4520f9cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:57.333432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1262/2542] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:49:57.337982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c191c-fe63-405e-bacf-39ae15e17aea None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:57.348016 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c191c-fe63-405e-bacf-39ae15e17aea None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:57.353792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b08d94-c544-4d22-9cad-4842bc3d5c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:57.354402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b08d94-c544-4d22-9cad-4842bc3d5c73 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:57.354705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b08d94-c544-4d22-9cad-4842bc3d5c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:57.354974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b08d94-c544-4d22-9cad-4842bc3d5c73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:57.355614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b08d94-c544-4d22-9cad-4842bc3d5c73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:49:57.355916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b08d94-c544-4d22-9cad-4842bc3d5c73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:57.357533 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b08d94-c544-4d22-9cad-4842bc3d5c73 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:57.371256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1281/2543] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:49:57.376005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94ea84-eb48-4c02-9da2-7da4520f9cef None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:57.386440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94ea84-eb48-4c02-9da2-7da4520f9cef None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:57.391110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ca98f-58e0-423d-b4ba-3976432dd0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:57.391716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ca98f-58e0-423d-b4ba-3976432dd0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:57.391999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ca98f-58e0-423d-b4ba-3976432dd0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:57.392272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ca98f-58e0-423d-b4ba-3976432dd0c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:57.392916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ca98f-58e0-423d-b4ba-3976432dd0c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:57.395806 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ca98f-58e0-423d-b4ba-3976432dd0c7 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:57.427656 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1263/2544] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:49:57 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:57.432339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b08d94-c544-4d22-9cad-4842bc3d5c73 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:57.443052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b08d94-c544-4d22-9cad-4842bc3d5c73 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:57.505639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df13eb1-2952-425d-9917-f96285b1d026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:57.506249 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df13eb1-2952-425d-9917-f96285b1d026 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:57.506531 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df13eb1-2952-425d-9917-f96285b1d026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:57.506799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df13eb1-2952-425d-9917-f96285b1d026 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:57.507426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df13eb1-2952-425d-9917-f96285b1d026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:57.509361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df13eb1-2952-425d-9917-f96285b1d026 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:57.534355 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1282/2545] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:49:57 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:57.535602 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ca98f-58e0-423d-b4ba-3976432dd0c7 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:57.554875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ca98f-58e0-423d-b4ba-3976432dd0c7 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:57.559907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca25c9-9fb7-4af8-a76e-5497894d64d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:57.560941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca25c9-9fb7-4af8-a76e-5497894d64d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:57.560941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca25c9-9fb7-4af8-a76e-5497894d64d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:57.561191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca25c9-9fb7-4af8-a76e-5497894d64d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:57.562460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca25c9-9fb7-4af8-a76e-5497894d64d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:57.563379 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca25c9-9fb7-4af8-a76e-5497894d64d5 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:57.574760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1264/2546] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:49:57.578989 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df13eb1-2952-425d-9917-f96285b1d026 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:57.590209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df13eb1-2952-425d-9917-f96285b1d026 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:57.595374 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6c659-078f-4480-99b3-2a169c5724dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:57.595986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6c659-078f-4480-99b3-2a169c5724dd admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:57.596429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6c659-078f-4480-99b3-2a169c5724dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:57.596737 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6c659-078f-4480-99b3-2a169c5724dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:57.597667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6c659-078f-4480-99b3-2a169c5724dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:57.600090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6c659-078f-4480-99b3-2a169c5724dd admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:57.612166 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1283/2547] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:49:57.616183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca25c9-9fb7-4af8-a76e-5497894d64d5 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:57.626433 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca25c9-9fb7-4af8-a76e-5497894d64d5 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:57.631378 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f75ea-2a23-4c54-8e09-83c2be3d92f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:57.632061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f75ea-2a23-4c54-8e09-83c2be3d92f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:57.632340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f75ea-2a23-4c54-8e09-83c2be3d92f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:57.632612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f75ea-2a23-4c54-8e09-83c2be3d92f9 admin admin] PATH_INFO: `/v3/projects/17965bf69c104b2cafb63891739d4fb5/users/cb400f28ed4c49d59ec9fd414ce2944a/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:57.648173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f75ea-2a23-4c54-8e09-83c2be3d92f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17965bf69c104b2cafb63891739d4fb5, user_id=cb400f28ed4c49d59ec9fd414ce2944a, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:57.650858 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f75ea-2a23-4c54-8e09-83c2be3d92f9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:57.664874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1265/2548] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:49:57 2022] PUT /identity/v3/projects/17965bf69c104b2cafb63891739d4fb5/users/cb400f28ed4c49d59ec9fd414ce2944a/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:57.676011 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebd0fc-4cf2-4999-93d3-18de49c34136 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:57.676298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebd0fc-4cf2-4999-93d3-18de49c34136 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:57.676574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebd0fc-4cf2-4999-93d3-18de49c34136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:57.704169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-67ebd0fc-4cf2-4999-93d3-18de49c34136 None None] MFA Rules not processed for user `cb400f28ed4c49d59ec9fd414ce2944a`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:57.705401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-67ebd0fc-4cf2-4999-93d3-18de49c34136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:57.770847 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1284/2549] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:57.782074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f75ea-2a23-4c54-8e09-83c2be3d92f9 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:57.783759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f41f75ea-2a23-4c54-8e09-83c2be3d92f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:57.806092 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:57.844734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f75ea-2a23-4c54-8e09-83c2be3d92f9 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:57.848701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023aa5d-56b2-4866-8f28-b9e89b7d2d87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:57.849342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023aa5d-56b2-4866-8f28-b9e89b7d2d87 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:57.849624 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023aa5d-56b2-4866-8f28-b9e89b7d2d87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:57.849898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023aa5d-56b2-4866-8f28-b9e89b7d2d87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:57.858142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023aa5d-56b2-4866-8f28-b9e89b7d2d87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:57.859564 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023aa5d-56b2-4866-8f28-b9e89b7d2d87 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:57.868483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1266/2550] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:58.258859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebd0fc-4cf2-4999-93d3-18de49c34136 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:58.260336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-67ebd0fc-4cf2-4999-93d3-18de49c34136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:58.280410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:58.328206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebd0fc-4cf2-4999-93d3-18de49c34136 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:58.332761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:58.333424 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:58.333707 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:58.333981 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] PATH_INFO: `/v3/users/7e1670c075894ec0b0c66797124912ce` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:58.335102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e1670c075894ec0b0c66797124912ce)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:58.337006 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:58.383474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e1670c075894ec0b0c66797124912ce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:58.395839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e1670c075894ec0b0c66797124912ce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:58.396592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e1670c075894ec0b0c66797124912ce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:58.403768 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e1670c075894ec0b0c66797124912ce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:58.404411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e1670c075894ec0b0c66797124912ce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:58.417842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1285/2551] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:49:58 2022] DELETE /identity/v3/users/7e1670c075894ec0b0c66797124912ce => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:58.422815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab0729-efd0-442c-ad0c-fa37bf4c6cb4 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:58.423105 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab0729-efd0-442c-ad0c-fa37bf4c6cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:58.423383 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab0729-efd0-442c-ad0c-fa37bf4c6cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:58.442817 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-5cab0729-efd0-442c-ad0c-fa37bf4c6cb4 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:58.444015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5cab0729-efd0-442c-ad0c-fa37bf4c6cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:58.502723 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1267/2552] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:58.513682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:58.515718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:58.573394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07489e91-8409-4a55-a9da-c2ceff7436f7 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:58.577276 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88594c-5014-4be7-843c-063e0ee285cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:58.577897 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88594c-5014-4be7-843c-063e0ee285cd service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:58.578181 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88594c-5014-4be7-843c-063e0ee285cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:58.578463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88594c-5014-4be7-843c-063e0ee285cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:58.588185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88594c-5014-4be7-843c-063e0ee285cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:58.589639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88594c-5014-4be7-843c-063e0ee285cd service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:58.599799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1286/2553] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:49:58.865420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab0729-efd0-442c-ad0c-fa37bf4c6cb4 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:58.866901 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5cab0729-efd0-442c-ad0c-fa37bf4c6cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:58.879386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab0729-efd0-442c-ad0c-fa37bf4c6cb4 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:58.884040 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:58.884675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:58.884956 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:58.885267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] PATH_INFO: `/v3/projects/f471fbe510124f489638c4bc3a02c35d` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:58.886390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f471fbe510124f489638c4bc3a02c35d)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:58.888303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:58.936178 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f471fbe510124f489638c4bc3a02c35d', 'request_id': 'req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:58.942160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f471fbe510124f489638c4bc3a02c35d', 'request_id': 'req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:58.949884 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f471fbe510124f489638c4bc3a02c35d', 'request_id': 'req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:58.963865 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] The token cache is being invalidate because project f471fbe510124f489638c4bc3a02c35d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:49:58.964160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:58.964809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:58.966321 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1268/2554] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:49:58 2022] DELETE /identity/v3/projects/f471fbe510124f489638c4bc3a02c35d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:58.970387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88594c-5014-4be7-843c-063e0ee285cd service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:58.980744 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88594c-5014-4be7-843c-063e0ee285cd service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:59.019791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:59.041706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c511a88-542d-4520-aa92-9285fb9d8a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:59.042333 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c511a88-542d-4520-aa92-9285fb9d8a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:59.042609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c511a88-542d-4520-aa92-9285fb9d8a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:59.042879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c511a88-542d-4520-aa92-9285fb9d8a1f admin admin] PATH_INFO: `/v3/users/5d4f7a5fb99048bf86b5b231dd1b7c9f` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:59.043986 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c511a88-542d-4520-aa92-9285fb9d8a1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d4f7a5fb99048bf86b5b231dd1b7c9f)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:59.046182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c511a88-542d-4520-aa92-9285fb9d8a1f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:59.092328 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9c511a88-542d-4520-aa92-9285fb9d8a1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d4f7a5fb99048bf86b5b231dd1b7c9f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:59.104847 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9c511a88-542d-4520-aa92-9285fb9d8a1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d4f7a5fb99048bf86b5b231dd1b7c9f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:59.105541 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9c511a88-542d-4520-aa92-9285fb9d8a1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d4f7a5fb99048bf86b5b231dd1b7c9f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:59.112372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9c511a88-542d-4520-aa92-9285fb9d8a1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d4f7a5fb99048bf86b5b231dd1b7c9f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:59.113002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9c511a88-542d-4520-aa92-9285fb9d8a1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d4f7a5fb99048bf86b5b231dd1b7c9f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:59.126856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1287/2555] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:49:58 2022] DELETE /identity/v3/users/5d4f7a5fb99048bf86b5b231dd1b7c9f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:49:59.418309 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:59.419842 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:59.445477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56e2f5-4bb8-4e03-98b5-f2ef0b3eae85 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:59.503020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:59.529692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d044e5b-098e-48f8-922c-991b13adbf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:59.530360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d044e5b-098e-48f8-922c-991b13adbf58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:59.530676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d044e5b-098e-48f8-922c-991b13adbf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:59.530917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d044e5b-098e-48f8-922c-991b13adbf58 admin admin] PATH_INFO: `/v3/projects/15e2fa0fa67d48e6bee08ca45a071cec` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:59.532060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d044e5b-098e-48f8-922c-991b13adbf58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e2fa0fa67d48e6bee08ca45a071cec)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:59.534039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d044e5b-098e-48f8-922c-991b13adbf58 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:59.583519 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8d044e5b-098e-48f8-922c-991b13adbf58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e2fa0fa67d48e6bee08ca45a071cec', 'request_id': 'req-8d044e5b-098e-48f8-922c-991b13adbf58'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:59.589117 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8d044e5b-098e-48f8-922c-991b13adbf58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e2fa0fa67d48e6bee08ca45a071cec', 'request_id': 'req-8d044e5b-098e-48f8-922c-991b13adbf58'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:59.597053 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8d044e5b-098e-48f8-922c-991b13adbf58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e2fa0fa67d48e6bee08ca45a071cec', 'request_id': 'req-8d044e5b-098e-48f8-922c-991b13adbf58'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:59.608795 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8d044e5b-098e-48f8-922c-991b13adbf58 admin admin] The token cache is being invalidate because project 15e2fa0fa67d48e6bee08ca45a071cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:49:59.609083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8d044e5b-098e-48f8-922c-991b13adbf58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:59.609778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8d044e5b-098e-48f8-922c-991b13adbf58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:49:59.611551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1269/2556] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:49:59 2022] DELETE /identity/v3/projects/15e2fa0fa67d48e6bee08ca45a071cec => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:49:59.617984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d8499-d4fb-4a0a-b050-f810f72ab10f None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:59.618541 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d8499-d4fb-4a0a-b050-f810f72ab10f None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:59.618688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d8499-d4fb-4a0a-b050-f810f72ab10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:59.639487 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f88d8499-d4fb-4a0a-b050-f810f72ab10f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:59.640685 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f88d8499-d4fb-4a0a-b050-f810f72ab10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:59.704031 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1288/2557] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:59.712194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06bd98-290e-49af-8d13-d02f873119be None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:59.712514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06bd98-290e-49af-8d13-d02f873119be None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:59.712864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06bd98-290e-49af-8d13-d02f873119be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:59.735151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-9d06bd98-290e-49af-8d13-d02f873119be None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:49:59.736286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9d06bd98-290e-49af-8d13-d02f873119be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:49:59.744941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1270/2558] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:49:59.749442 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d8499-d4fb-4a0a-b050-f810f72ab10f None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:59.760269 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d8499-d4fb-4a0a-b050-f810f72ab10f None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:59.764892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86ad52-fd50-4652-9cec-6ea7fa77feab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:59.765539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86ad52-fd50-4652-9cec-6ea7fa77feab admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:59.765824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86ad52-fd50-4652-9cec-6ea7fa77feab admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:59.766102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86ad52-fd50-4652-9cec-6ea7fa77feab admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:59.766753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86ad52-fd50-4652-9cec-6ea7fa77feab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:49:59.767062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86ad52-fd50-4652-9cec-6ea7fa77feab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:59.769053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86ad52-fd50-4652-9cec-6ea7fa77feab admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:59.780779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1289/2559] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:49:59.785371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06bd98-290e-49af-8d13-d02f873119be None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:59.794653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06bd98-290e-49af-8d13-d02f873119be None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:59.799607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88145921-ef1a-4b14-b180-4df5d08f30e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:59.800198 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88145921-ef1a-4b14-b180-4df5d08f30e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:59.800477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88145921-ef1a-4b14-b180-4df5d08f30e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:59.800747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88145921-ef1a-4b14-b180-4df5d08f30e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:59.801940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88145921-ef1a-4b14-b180-4df5d08f30e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:59.803828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88145921-ef1a-4b14-b180-4df5d08f30e4 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:59.832961 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1271/2560] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:49:59 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:59.837195 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86ad52-fd50-4652-9cec-6ea7fa77feab admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:59.847191 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86ad52-fd50-4652-9cec-6ea7fa77feab admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:59.893415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:49:59.907880 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498efa22-f166-4078-8678-90fe53326639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:59.908519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498efa22-f166-4078-8678-90fe53326639 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:59.908808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498efa22-f166-4078-8678-90fe53326639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:59.909151 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498efa22-f166-4078-8678-90fe53326639 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:59.909811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498efa22-f166-4078-8678-90fe53326639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:59.912258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498efa22-f166-4078-8678-90fe53326639 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:59.933669 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1290/2561] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:49:59 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:49:59.937943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88145921-ef1a-4b14-b180-4df5d08f30e4 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:59.948394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88145921-ef1a-4b14-b180-4df5d08f30e4 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:59.953073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac16d9-c807-4145-96c3-742428925172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:59.953705 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac16d9-c807-4145-96c3-742428925172 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:59.953981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac16d9-c807-4145-96c3-742428925172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:59.954246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac16d9-c807-4145-96c3-742428925172 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:59.954905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac16d9-c807-4145-96c3-742428925172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:59.956474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac16d9-c807-4145-96c3-742428925172 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:49:59.966720 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1272/2562] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:49:59.970773 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498efa22-f166-4078-8678-90fe53326639 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:49:59.981726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498efa22-f166-4078-8678-90fe53326639 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:49:59.986411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c94dd5-0ebf-429e-b8b8-b2f077a6a672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:49:59.986973 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c94dd5-0ebf-429e-b8b8-b2f077a6a672 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:49:59.987249 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c94dd5-0ebf-429e-b8b8-b2f077a6a672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:49:59.987519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c94dd5-0ebf-429e-b8b8-b2f077a6a672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:49:59.988152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c94dd5-0ebf-429e-b8b8-b2f077a6a672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:49:59.990277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c94dd5-0ebf-429e-b8b8-b2f077a6a672 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:00.120840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1291/2563] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:00.125210 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac16d9-c807-4145-96c3-742428925172 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:00.135335 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac16d9-c807-4145-96c3-742428925172 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:00.141482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588026b-bf98-4a8d-b38e-a587d13dc081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:00.142136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588026b-bf98-4a8d-b38e-a587d13dc081 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:00.142412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588026b-bf98-4a8d-b38e-a587d13dc081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:00.142685 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588026b-bf98-4a8d-b38e-a587d13dc081 admin admin] PATH_INFO: `/v3/projects/63bb9bb9921041d49d13658fb2218f2e/users/377ff79677314498a2b7b1500ef2f069/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:00.157152 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588026b-bf98-4a8d-b38e-a587d13dc081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63bb9bb9921041d49d13658fb2218f2e, user_id=377ff79677314498a2b7b1500ef2f069, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:00.159774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588026b-bf98-4a8d-b38e-a587d13dc081 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:00.173362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1273/2564] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:00 2022] PUT /identity/v3/projects/63bb9bb9921041d49d13658fb2218f2e/users/377ff79677314498a2b7b1500ef2f069/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:00.185876 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f680b0-3a76-4fa2-a849-ddaf11ace354 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:00.186160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f680b0-3a76-4fa2-a849-ddaf11ace354 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:00.186436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f680b0-3a76-4fa2-a849-ddaf11ace354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:00.213349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-d7f680b0-3a76-4fa2-a849-ddaf11ace354 None None] MFA Rules not processed for user `377ff79677314498a2b7b1500ef2f069`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:00.214562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d7f680b0-3a76-4fa2-a849-ddaf11ace354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:00.287328 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1292/2565] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:00.291888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588026b-bf98-4a8d-b38e-a587d13dc081 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:00.353474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588026b-bf98-4a8d-b38e-a587d13dc081 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:00.358166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba59ce-1f80-4a59-aa5a-29552e3ef9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:00.358765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba59ce-1f80-4a59-aa5a-29552e3ef9db admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:00.359044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba59ce-1f80-4a59-aa5a-29552e3ef9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:00.359316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba59ce-1f80-4a59-aa5a-29552e3ef9db admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:00.359946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba59ce-1f80-4a59-aa5a-29552e3ef9db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:00.361856 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba59ce-1f80-4a59-aa5a-29552e3ef9db admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:00.382571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:00.390703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1274/2566] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:00 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:00.395270 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f680b0-3a76-4fa2-a849-ddaf11ace354 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:00.405873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f680b0-3a76-4fa2-a849-ddaf11ace354 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:00.462287 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1b4b5-247a-4abe-8ec0-8fb0d118c268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:00.462905 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1b4b5-247a-4abe-8ec0-8fb0d118c268 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:00.463182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1b4b5-247a-4abe-8ec0-8fb0d118c268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:00.463459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1b4b5-247a-4abe-8ec0-8fb0d118c268 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:00.464133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1b4b5-247a-4abe-8ec0-8fb0d118c268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:00.466088 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1b4b5-247a-4abe-8ec0-8fb0d118c268 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:00.484387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba59ce-1f80-4a59-aa5a-29552e3ef9db admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:00.486145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-01ba59ce-1f80-4a59-aa5a-29552e3ef9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:00.492414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1293/2567] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:00 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:00.494477 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1b4b5-247a-4abe-8ec0-8fb0d118c268 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:00.499181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba59ce-1f80-4a59-aa5a-29552e3ef9db admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:00.503887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c65b4-a159-4b0a-b38b-5dfa3f2d64d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:00.504152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1b4b5-247a-4abe-8ec0-8fb0d118c268 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:00.504517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c65b4-a159-4b0a-b38b-5dfa3f2d64d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:00.504801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c65b4-a159-4b0a-b38b-5dfa3f2d64d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:00.505073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c65b4-a159-4b0a-b38b-5dfa3f2d64d5 admin admin] PATH_INFO: `/v3/users/cb400f28ed4c49d59ec9fd414ce2944a` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:00.506248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c65b4-a159-4b0a-b38b-5dfa3f2d64d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb400f28ed4c49d59ec9fd414ce2944a)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:00.508157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c65b4-a159-4b0a-b38b-5dfa3f2d64d5 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:00.508885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c91b2-c3a7-4fa2-ac53-32a9c0b2eff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:00.509515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c91b2-c3a7-4fa2-ac53-32a9c0b2eff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:00.509805 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c91b2-c3a7-4fa2-ac53-32a9c0b2eff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:00.510071 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c91b2-c3a7-4fa2-ac53-32a9c0b2eff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:00.510719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c91b2-c3a7-4fa2-ac53-32a9c0b2eff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:00.512292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c91b2-c3a7-4fa2-ac53-32a9c0b2eff9 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:00.536454 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1294/2568] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:00.539206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c91b2-c3a7-4fa2-ac53-32a9c0b2eff9 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:00.550520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c91b2-c3a7-4fa2-ac53-32a9c0b2eff9 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:00.555478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e56830-b2b8-417d-9632-86847c5611cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:00.556136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e56830-b2b8-417d-9632-86847c5611cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:00.556416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e56830-b2b8-417d-9632-86847c5611cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:00.556694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e56830-b2b8-417d-9632-86847c5611cf admin admin] PATH_INFO: `/v3/projects/cb6ae49ff8cf4cdc8d58ce43523d0558/users/1a48727e15e64f36a0a8d02cf0e42a0d/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:00.565159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b11c65b4-a159-4b0a-b38b-5dfa3f2d64d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb400f28ed4c49d59ec9fd414ce2944a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:00.572470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b11c65b4-a159-4b0a-b38b-5dfa3f2d64d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb400f28ed4c49d59ec9fd414ce2944a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:00.572974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e56830-b2b8-417d-9632-86847c5611cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6ae49ff8cf4cdc8d58ce43523d0558, user_id=1a48727e15e64f36a0a8d02cf0e42a0d, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:00.573208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b11c65b4-a159-4b0a-b38b-5dfa3f2d64d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb400f28ed4c49d59ec9fd414ce2944a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:00.573837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b11c65b4-a159-4b0a-b38b-5dfa3f2d64d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb400f28ed4c49d59ec9fd414ce2944a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:00.576001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e56830-b2b8-417d-9632-86847c5611cf admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:00.581861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b11c65b4-a159-4b0a-b38b-5dfa3f2d64d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb400f28ed4c49d59ec9fd414ce2944a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:00.590615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1295/2569] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:00 2022] PUT /identity/v3/projects/cb6ae49ff8cf4cdc8d58ce43523d0558/users/1a48727e15e64f36a0a8d02cf0e42a0d/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:00.594859 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e56830-b2b8-417d-9632-86847c5611cf admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:00.596532 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f1e56830-b2b8-417d-9632-86847c5611cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:00.601565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1275/2570] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:00 2022] DELETE /identity/v3/users/cb400f28ed4c49d59ec9fd414ce2944a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:00.605965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6b361-d68f-4c63-8a7a-33a3f27952ba None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:00.606256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6b361-d68f-4c63-8a7a-33a3f27952ba None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:00.606533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6b361-d68f-4c63-8a7a-33a3f27952ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:00.610795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:00.625787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-a4a6b361-d68f-4c63-8a7a-33a3f27952ba None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:00.627010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a4a6b361-d68f-4c63-8a7a-33a3f27952ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:00.667216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e56830-b2b8-417d-9632-86847c5611cf admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:00.687863 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1276/2571] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:00.698230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6b361-d68f-4c63-8a7a-33a3f27952ba None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:00.699642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a4a6b361-d68f-4c63-8a7a-33a3f27952ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:00.725825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc30a1-afb8-45bf-a859-5fcb9343d142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:00.726437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc30a1-afb8-45bf-a859-5fcb9343d142 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:00.726720 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc30a1-afb8-45bf-a859-5fcb9343d142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:00.727001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc30a1-afb8-45bf-a859-5fcb9343d142 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:00.727644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc30a1-afb8-45bf-a859-5fcb9343d142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:00.729268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc30a1-afb8-45bf-a859-5fcb9343d142 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:00.739582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1296/2572] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:00.743490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc30a1-afb8-45bf-a859-5fcb9343d142 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:00.753527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc30a1-afb8-45bf-a859-5fcb9343d142 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:00.758762 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d205ab8-3077-4f54-bc9b-d28aace5749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:00.759411 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d205ab8-3077-4f54-bc9b-d28aace5749f admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:00.759693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d205ab8-3077-4f54-bc9b-d28aace5749f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:00.759968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d205ab8-3077-4f54-bc9b-d28aace5749f admin admin] PATH_INFO: `/v3/projects/cb6ae49ff8cf4cdc8d58ce43523d0558/users/1a48727e15e64f36a0a8d02cf0e42a0d/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:00.761929 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d205ab8-3077-4f54-bc9b-d28aace5749f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6ae49ff8cf4cdc8d58ce43523d0558, user_id=1a48727e15e64f36a0a8d02cf0e42a0d, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:00.762049 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6b361-d68f-4c63-8a7a-33a3f27952ba None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:00.764542 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d205ab8-3077-4f54-bc9b-d28aace5749f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:00.765717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb43ab8-8db3-40fc-9f06-708ccd8f303e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:00.766318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb43ab8-8db3-40fc-9f06-708ccd8f303e service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:00.766597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb43ab8-8db3-40fc-9f06-708ccd8f303e service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:00.766926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb43ab8-8db3-40fc-9f06-708ccd8f303e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:00.776382 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1297/2573] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:00 2022] PUT /identity/v3/projects/cb6ae49ff8cf4cdc8d58ce43523d0558/users/1a48727e15e64f36a0a8d02cf0e42a0d/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:00.781272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb43ab8-8db3-40fc-9f06-708ccd8f303e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:00.781272 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb43ab8-8db3-40fc-9f06-708ccd8f303e service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:00.786847 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb262b15-33b1-4e03-8812-5cd9a170ee4d None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:00.787150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb262b15-33b1-4e03-8812-5cd9a170ee4d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:00.787416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb262b15-33b1-4e03-8812-5cd9a170ee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:00.790886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1277/2574] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:50:00.816078 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-bb262b15-33b1-4e03-8812-5cd9a170ee4d None None] MFA Rules not processed for user `1a48727e15e64f36a0a8d02cf0e42a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:00.817886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-bb262b15-33b1-4e03-8812-5cd9a170ee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:00.894000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1298/2575] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:00.905286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb43ab8-8db3-40fc-9f06-708ccd8f303e service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:00.906752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7cb43ab8-8db3-40fc-9f06-708ccd8f303e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:00.921844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:00.991412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb43ab8-8db3-40fc-9f06-708ccd8f303e service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:01.041447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5899e-d6b0-48a2-be58-090062d3e3a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:01.041967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5899e-d6b0-48a2-be58-090062d3e3a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:01.042241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5899e-d6b0-48a2-be58-090062d3e3a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:01.042507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5899e-d6b0-48a2-be58-090062d3e3a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:01.044374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1fd5899e-d6b0-48a2-be58-090062d3e3a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:01.070711 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb262b15-33b1-4e03-8812-5cd9a170ee4d None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:01.072341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-bb262b15-33b1-4e03-8812-5cd9a170ee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:01.103787 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:01.117820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5899e-d6b0-48a2-be58-090062d3e3a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:01.119260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5899e-d6b0-48a2-be58-090062d3e3a0 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:01.128395 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1278/2576] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:01.144719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb262b15-33b1-4e03-8812-5cd9a170ee4d None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:01.149390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3973762-8bda-454b-b712-4adf17952b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:01.150013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3973762-8bda-454b-b712-4adf17952b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:01.150285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3973762-8bda-454b-b712-4adf17952b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:01.150554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3973762-8bda-454b-b712-4adf17952b04 admin admin] PATH_INFO: `/v3/projects/17965bf69c104b2cafb63891739d4fb5` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:01.151676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3973762-8bda-454b-b712-4adf17952b04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17965bf69c104b2cafb63891739d4fb5)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:01.153586 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3973762-8bda-454b-b712-4adf17952b04 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:01.204975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d3973762-8bda-454b-b712-4adf17952b04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17965bf69c104b2cafb63891739d4fb5', 'request_id': 'req-d3973762-8bda-454b-b712-4adf17952b04'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:01.210992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d3973762-8bda-454b-b712-4adf17952b04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17965bf69c104b2cafb63891739d4fb5', 'request_id': 'req-d3973762-8bda-454b-b712-4adf17952b04'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:01.217491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d3973762-8bda-454b-b712-4adf17952b04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17965bf69c104b2cafb63891739d4fb5', 'request_id': 'req-d3973762-8bda-454b-b712-4adf17952b04'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:01.230823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d3973762-8bda-454b-b712-4adf17952b04 admin admin] The token cache is being invalidate because project 17965bf69c104b2cafb63891739d4fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:01.231295 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d3973762-8bda-454b-b712-4adf17952b04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:01.231923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d3973762-8bda-454b-b712-4adf17952b04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:01.233604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1299/2577] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:01 2022] DELETE /identity/v3/projects/17965bf69c104b2cafb63891739d4fb5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:01.238940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c38787-b97c-469d-9ca2-1c84069cb6d5 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:01.239231 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c38787-b97c-469d-9ca2-1c84069cb6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:01.239509 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c38787-b97c-469d-9ca2-1c84069cb6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:01.262852 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-79c38787-b97c-469d-9ca2-1c84069cb6d5 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:01.264143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-79c38787-b97c-469d-9ca2-1c84069cb6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:01.327634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1279/2578] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:01.335412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31e602-6be8-4478-a025-8f068723abeb None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:01.335717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31e602-6be8-4478-a025-8f068723abeb None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:01.335967 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31e602-6be8-4478-a025-8f068723abeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:01.359106 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-4f31e602-6be8-4478-a025-8f068723abeb None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:01.360681 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4f31e602-6be8-4478-a025-8f068723abeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:01.371726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1300/2579] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:01.376025 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c38787-b97c-469d-9ca2-1c84069cb6d5 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:01.387857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c38787-b97c-469d-9ca2-1c84069cb6d5 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:01.393392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d26a1e-9a52-4ad4-b20c-1a8417ffb904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:01.394007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d26a1e-9a52-4ad4-b20c-1a8417ffb904 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:01.394289 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d26a1e-9a52-4ad4-b20c-1a8417ffb904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:01.394565 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d26a1e-9a52-4ad4-b20c-1a8417ffb904 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:01.395211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d26a1e-9a52-4ad4-b20c-1a8417ffb904 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:50:01.395516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d26a1e-9a52-4ad4-b20c-1a8417ffb904 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:01.397138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d26a1e-9a52-4ad4-b20c-1a8417ffb904 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:01.411641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1280/2580] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:50:01.416069 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31e602-6be8-4478-a025-8f068723abeb None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:01.430679 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31e602-6be8-4478-a025-8f068723abeb None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:01.444161 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee75433-9e40-4f96-be91-facbeda2564b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:01.445161 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee75433-9e40-4f96-be91-facbeda2564b admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:01.445161 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee75433-9e40-4f96-be91-facbeda2564b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:01.445346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee75433-9e40-4f96-be91-facbeda2564b admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:01.445968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee75433-9e40-4f96-be91-facbeda2564b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:01.447835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee75433-9e40-4f96-be91-facbeda2564b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:01.481031 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1301/2581] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:01 2022] POST /identity/v3/projects => generated 399 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:50:01.485366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d26a1e-9a52-4ad4-b20c-1a8417ffb904 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:01.496595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d26a1e-9a52-4ad4-b20c-1a8417ffb904 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:01.504314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:01.560781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c086652-b14e-4667-b381-86b2528b2624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:01.561332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c086652-b14e-4667-b381-86b2528b2624 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:01.561610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c086652-b14e-4667-b381-86b2528b2624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:01.561878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c086652-b14e-4667-b381-86b2528b2624 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:01.562497 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c086652-b14e-4667-b381-86b2528b2624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:01.564394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c086652-b14e-4667-b381-86b2528b2624 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:01.585594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1281/2582] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:01 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:01.589863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee75433-9e40-4f96-be91-facbeda2564b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:01.600984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee75433-9e40-4f96-be91-facbeda2564b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:01.605882 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc68f05-bf43-4019-a588-2f5b6e082a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:01.606495 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc68f05-bf43-4019-a588-2f5b6e082a04 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:01.606760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc68f05-bf43-4019-a588-2f5b6e082a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:01.607025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc68f05-bf43-4019-a588-2f5b6e082a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:01.607662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc68f05-bf43-4019-a588-2f5b6e082a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:01.609263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc68f05-bf43-4019-a588-2f5b6e082a04 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:01.620974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1302/2583] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:01.624950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c086652-b14e-4667-b381-86b2528b2624 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:01.634882 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c086652-b14e-4667-b381-86b2528b2624 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:01.639460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31caaf7-0adc-48d0-bfef-9829358cee51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:01.640055 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31caaf7-0adc-48d0-bfef-9829358cee51 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:01.640323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31caaf7-0adc-48d0-bfef-9829358cee51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:01.640596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31caaf7-0adc-48d0-bfef-9829358cee51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:01.641333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31caaf7-0adc-48d0-bfef-9829358cee51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:01.642909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31caaf7-0adc-48d0-bfef-9829358cee51 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:01.653949 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1282/2584] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:01.658365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc68f05-bf43-4019-a588-2f5b6e082a04 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:01.669842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc68f05-bf43-4019-a588-2f5b6e082a04 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:01.675032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9eef6-a73b-49c4-83b8-0b276a639e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:01.675694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9eef6-a73b-49c4-83b8-0b276a639e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:01.676660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9eef6-a73b-49c4-83b8-0b276a639e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:01.676938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9eef6-a73b-49c4-83b8-0b276a639e78 admin admin] PATH_INFO: `/v3/projects/328068e1e5804dc49ec4979166b42093/users/c3cd72b6255e4a7ba140c87f1ed99d18/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:01.692140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9eef6-a73b-49c4-83b8-0b276a639e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=328068e1e5804dc49ec4979166b42093, user_id=c3cd72b6255e4a7ba140c87f1ed99d18, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:01.694879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9eef6-a73b-49c4-83b8-0b276a639e78 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:01.711296 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1303/2585] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:01 2022] PUT /identity/v3/projects/328068e1e5804dc49ec4979166b42093/users/c3cd72b6255e4a7ba140c87f1ed99d18/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:01.721987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd0561-c476-44ab-bdef-6e133eb5dae8 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:01.722275 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd0561-c476-44ab-bdef-6e133eb5dae8 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:01.722581 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd0561-c476-44ab-bdef-6e133eb5dae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:01.752440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-5fbd0561-c476-44ab-bdef-6e133eb5dae8 None None] MFA Rules not processed for user `c3cd72b6255e4a7ba140c87f1ed99d18`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:01.753699 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5fbd0561-c476-44ab-bdef-6e133eb5dae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:01.826914 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1283/2586] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:01.839202 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9eef6-a73b-49c4-83b8-0b276a639e78 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:01.841062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b6f9eef6-a73b-49c4-83b8-0b276a639e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:01.900767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9eef6-a73b-49c4-83b8-0b276a639e78 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:01.904606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca602c-2797-42cc-a8fc-69d223882b12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:01.905266 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca602c-2797-42cc-a8fc-69d223882b12 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:01.905540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca602c-2797-42cc-a8fc-69d223882b12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:01.905816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca602c-2797-42cc-a8fc-69d223882b12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:01.911702 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:01.916515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca602c-2797-42cc-a8fc-69d223882b12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:01.918427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca602c-2797-42cc-a8fc-69d223882b12 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:01.928288 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1304/2587] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:06.808713 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd0561-c476-44ab-bdef-6e133eb5dae8 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:06.818321 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbd0561-c476-44ab-bdef-6e133eb5dae8 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:06.821979 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bea0ba-0485-4359-84ec-eea72e124cb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:06.822577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bea0ba-0485-4359-84ec-eea72e124cb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:06.822853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bea0ba-0485-4359-84ec-eea72e124cb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:06.823125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bea0ba-0485-4359-84ec-eea72e124cb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:06.824855 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f7bea0ba-0485-4359-84ec-eea72e124cb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:06.888423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bea0ba-0485-4359-84ec-eea72e124cb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:06.889895 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bea0ba-0485-4359-84ec-eea72e124cb6 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:06.899400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1284/2588] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:11.924633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:16.812268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca602c-2797-42cc-a8fc-69d223882b12 service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:16.817740 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e3ca602c-2797-42cc-a8fc-69d223882b12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:16.895205 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:16.989332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca602c-2797-42cc-a8fc-69d223882b12 service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:16.999422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3fbc0-8901-4e8b-95cb-550b966ea741 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:17.000320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3fbc0-8901-4e8b-95cb-550b966ea741 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:17.000607 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3fbc0-8901-4e8b-95cb-550b966ea741 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:17.000929 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3fbc0-8901-4e8b-95cb-550b966ea741 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:17.003416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d5e3fbc0-8901-4e8b-95cb-550b966ea741 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:17.023036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3fbc0-8901-4e8b-95cb-550b966ea741 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:17.035274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3fbc0-8901-4e8b-95cb-550b966ea741 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:17.057781 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1305/2589] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:17.208466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bea0ba-0485-4359-84ec-eea72e124cb6 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:17.272645 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bea0ba-0485-4359-84ec-eea72e124cb6 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:17.277246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:17.277870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:17.278146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:17.278447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] PATH_INFO: `/v3/users/c3cd72b6255e4a7ba140c87f1ed99d18` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:17.279648 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3cd72b6255e4a7ba140c87f1ed99d18)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:17.281608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:17.330107 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3cd72b6255e4a7ba140c87f1ed99d18'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:17.337949 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3cd72b6255e4a7ba140c87f1ed99d18'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:17.338696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3cd72b6255e4a7ba140c87f1ed99d18'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:17.339316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3cd72b6255e4a7ba140c87f1ed99d18'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:17.348871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3cd72b6255e4a7ba140c87f1ed99d18'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:17.368470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1285/2590] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:17 2022] DELETE /identity/v3/users/c3cd72b6255e4a7ba140c87f1ed99d18 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:17.373112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f92c69-8a29-4eda-acf4-343edbb50998 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:17.373490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f92c69-8a29-4eda-acf4-343edbb50998 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:17.373835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f92c69-8a29-4eda-acf4-343edbb50998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:17.396760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-37f92c69-8a29-4eda-acf4-343edbb50998 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:17.397935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-37f92c69-8a29-4eda-acf4-343edbb50998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:17.473383 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1306/2591] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:17.481950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:17.483556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:17.552812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67926484-8312-4593-bc15-49e5bba01533 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:17.556616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92640260-be7c-45b3-b380-854d775a10f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:17.557915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92640260-be7c-45b3-b380-854d775a10f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:17.558200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92640260-be7c-45b3-b380-854d775a10f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:17.558479 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92640260-be7c-45b3-b380-854d775a10f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:17.568734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92640260-be7c-45b3-b380-854d775a10f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:17.570296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92640260-be7c-45b3-b380-854d775a10f7 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:17.580661 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1286/2592] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:17.895554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f92c69-8a29-4eda-acf4-343edbb50998 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:17.897167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-37f92c69-8a29-4eda-acf4-343edbb50998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:17.927303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f92c69-8a29-4eda-acf4-343edbb50998 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:17.933158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:17.936919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:17.937269 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:17.937509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf admin admin] PATH_INFO: `/v3/projects/328068e1e5804dc49ec4979166b42093` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:17.939252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=328068e1e5804dc49ec4979166b42093)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:17.941312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:18.001800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '328068e1e5804dc49ec4979166b42093', 'request_id': 'req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:18.007956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '328068e1e5804dc49ec4979166b42093', 'request_id': 'req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:18.014332 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '328068e1e5804dc49ec4979166b42093', 'request_id': 'req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:18.027641 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf admin admin] The token cache is being invalidate because project 328068e1e5804dc49ec4979166b42093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:18.027934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:18.028571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-0646b8b7-ce1c-49bf-8490-39aeb4379ecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:18.030130 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1307/2593] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:17 2022] DELETE /identity/v3/projects/328068e1e5804dc49ec4979166b42093 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:18.035575 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160937f7-3f1c-48f8-97ae-3ee4d30d40af None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:18.036032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160937f7-3f1c-48f8-97ae-3ee4d30d40af None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:18.036150 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160937f7-3f1c-48f8-97ae-3ee4d30d40af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:18.057475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-160937f7-3f1c-48f8-97ae-3ee4d30d40af None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:18.058692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-160937f7-3f1c-48f8-97ae-3ee4d30d40af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:18.082432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:18.129288 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1287/2594] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:18.138724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda84bed-17d5-4ab9-a43b-3b66a9157b2a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:18.139099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda84bed-17d5-4ab9-a43b-3b66a9157b2a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:18.139404 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda84bed-17d5-4ab9-a43b-3b66a9157b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:18.159977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-bda84bed-17d5-4ab9-a43b-3b66a9157b2a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:18.161456 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-bda84bed-17d5-4ab9-a43b-3b66a9157b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:18.169455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1308/2595] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:18.174042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160937f7-3f1c-48f8-97ae-3ee4d30d40af None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:18.184340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160937f7-3f1c-48f8-97ae-3ee4d30d40af None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:18.188965 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fdea6-427b-4f39-805b-2777ab298ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:18.189608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fdea6-427b-4f39-805b-2777ab298ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:18.189888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fdea6-427b-4f39-805b-2777ab298ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:18.190161 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fdea6-427b-4f39-805b-2777ab298ceb admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:18.190845 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fdea6-427b-4f39-805b-2777ab298ceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:50:18.191145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fdea6-427b-4f39-805b-2777ab298ceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:18.192747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fdea6-427b-4f39-805b-2777ab298ceb admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:18.204689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1288/2596] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:50:18.209429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda84bed-17d5-4ab9-a43b-3b66a9157b2a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:18.214075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:18.220426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda84bed-17d5-4ab9-a43b-3b66a9157b2a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:18.226572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3e806-e404-4126-a972-bf16c09e7df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:18.227184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3e806-e404-4126-a972-bf16c09e7df8 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:18.227462 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3e806-e404-4126-a972-bf16c09e7df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:18.227734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3e806-e404-4126-a972-bf16c09e7df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:18.228367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3e806-e404-4126-a972-bf16c09e7df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:18.230292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3e806-e404-4126-a972-bf16c09e7df8 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:18.261855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1309/2597] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:18 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:50:18.263077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fdea6-427b-4f39-805b-2777ab298ceb admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:18.273073 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fdea6-427b-4f39-805b-2777ab298ceb admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:18.340676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995234dd-a2db-41d4-9209-7aedc85a00b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:18.341423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995234dd-a2db-41d4-9209-7aedc85a00b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:18.341605 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995234dd-a2db-41d4-9209-7aedc85a00b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:18.341876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995234dd-a2db-41d4-9209-7aedc85a00b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:18.342534 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995234dd-a2db-41d4-9209-7aedc85a00b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:18.344501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995234dd-a2db-41d4-9209-7aedc85a00b0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:18.370229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1289/2598] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:18 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:18.371544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3e806-e404-4126-a972-bf16c09e7df8 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:18.382138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3e806-e404-4126-a972-bf16c09e7df8 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:18.386781 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b716c9-d5b8-4c18-b75f-b926d358e48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:18.387386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b716c9-d5b8-4c18-b75f-b926d358e48b admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:18.387664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b716c9-d5b8-4c18-b75f-b926d358e48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:18.387937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b716c9-d5b8-4c18-b75f-b926d358e48b admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:18.388582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b716c9-d5b8-4c18-b75f-b926d358e48b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:18.390197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b716c9-d5b8-4c18-b75f-b926d358e48b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:18.401146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1310/2599] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:18.405294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995234dd-a2db-41d4-9209-7aedc85a00b0 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:18.416317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995234dd-a2db-41d4-9209-7aedc85a00b0 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:18.421743 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2454b-90b9-43b6-b7d8-34ef44c0a06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:18.422353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2454b-90b9-43b6-b7d8-34ef44c0a06f admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:18.422633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2454b-90b9-43b6-b7d8-34ef44c0a06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:18.422908 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2454b-90b9-43b6-b7d8-34ef44c0a06f admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:18.423551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2454b-90b9-43b6-b7d8-34ef44c0a06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:18.425227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2454b-90b9-43b6-b7d8-34ef44c0a06f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:18.437176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1290/2600] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:18.441279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b716c9-d5b8-4c18-b75f-b926d358e48b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:18.453114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b716c9-d5b8-4c18-b75f-b926d358e48b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:18.458968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad15639-e558-40cf-82a2-7e575e1da67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:18.459190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad15639-e558-40cf-82a2-7e575e1da67c admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:18.460199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad15639-e558-40cf-82a2-7e575e1da67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:18.460199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad15639-e558-40cf-82a2-7e575e1da67c admin admin] PATH_INFO: `/v3/projects/ba777b9726e944a881c0953e605f7477/users/e4ecba03d9254112b343f36ade20754f/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:18.471460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2454b-90b9-43b6-b7d8-34ef44c0a06f admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:18.473829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-55c2454b-90b9-43b6-b7d8-34ef44c0a06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:18.475799 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad15639-e558-40cf-82a2-7e575e1da67c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba777b9726e944a881c0953e605f7477, user_id=e4ecba03d9254112b343f36ade20754f, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:18.478521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad15639-e558-40cf-82a2-7e575e1da67c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:18.492345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1311/2601] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:18 2022] PUT /identity/v3/projects/ba777b9726e944a881c0953e605f7477/users/e4ecba03d9254112b343f36ade20754f/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:18.497589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79c089-9bf4-4d88-9b42-357d547d842c None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:18.497857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79c089-9bf4-4d88-9b42-357d547d842c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:18.498133 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79c089-9bf4-4d88-9b42-357d547d842c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:18.516998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-8d79c089-9bf4-4d88-9b42-357d547d842c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:18.518279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8d79c089-9bf4-4d88-9b42-357d547d842c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:18.540084 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2454b-90b9-43b6-b7d8-34ef44c0a06f admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:18.583888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1312/2602] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:18.584431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:18.592706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d4f19-fe14-4c20-aa6a-0c0078e2491e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:18.593543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d4f19-fe14-4c20-aa6a-0c0078e2491e service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:18.594077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79c089-9bf4-4d88-9b42-357d547d842c None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:18.594390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d4f19-fe14-4c20-aa6a-0c0078e2491e service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:18.594796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d4f19-fe14-4c20-aa6a-0c0078e2491e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:18.596138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8d79c089-9bf4-4d88-9b42-357d547d842c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:18.598422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1d6d4f19-fe14-4c20-aa6a-0c0078e2491e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:18.659052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79c089-9bf4-4d88-9b42-357d547d842c None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:18.663922 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655d196-0a81-48d5-aed4-75c592290b8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:18.664528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655d196-0a81-48d5-aed4-75c592290b8b service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:18.664829 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655d196-0a81-48d5-aed4-75c592290b8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:18.665116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655d196-0a81-48d5-aed4-75c592290b8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:18.674717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655d196-0a81-48d5-aed4-75c592290b8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:18.676223 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655d196-0a81-48d5-aed4-75c592290b8b service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:18.685962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1313/2603] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:50:18.689578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d4f19-fe14-4c20-aa6a-0c0078e2491e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:18.691557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d4f19-fe14-4c20-aa6a-0c0078e2491e service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:18.711576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1291/2604] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:22.991442 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b618d4-1021-4156-b84c-a1d17a1f7460 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:22.991980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b618d4-1021-4156-b84c-a1d17a1f7460 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:22.991980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b618d4-1021-4156-b84c-a1d17a1f7460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:23.021659 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-99b618d4-1021-4156-b84c-a1d17a1f7460 None None] MFA Rules not processed for user `e4ecba03d9254112b343f36ade20754f`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:23.022804 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-99b618d4-1021-4156-b84c-a1d17a1f7460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:23.067369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:23.102491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1314/2605] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:23.104546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d4f19-fe14-4c20-aa6a-0c0078e2491e service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:23.120584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d4f19-fe14-4c20-aa6a-0c0078e2491e service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:23.133896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9f86b-ee9d-4fcf-b5cb-c93bc8d6a2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:23.133896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9f86b-ee9d-4fcf-b5cb-c93bc8d6a2c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:23.133896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9f86b-ee9d-4fcf-b5cb-c93bc8d6a2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:23.133896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9f86b-ee9d-4fcf-b5cb-c93bc8d6a2c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:23.134980 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9f86b-ee9d-4fcf-b5cb-c93bc8d6a2c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:23.136466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9f86b-ee9d-4fcf-b5cb-c93bc8d6a2c1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:23.191067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1292/2606] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:23 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:23.196488 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b618d4-1021-4156-b84c-a1d17a1f7460 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:23.210865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b618d4-1021-4156-b84c-a1d17a1f7460 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:23.281342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6c9b4-ccd5-47b8-bac4-28a79b8bbf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:23.281342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6c9b4-ccd5-47b8-bac4-28a79b8bbf05 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:23.281342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6c9b4-ccd5-47b8-bac4-28a79b8bbf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:23.281342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6c9b4-ccd5-47b8-bac4-28a79b8bbf05 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:23.281689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6c9b4-ccd5-47b8-bac4-28a79b8bbf05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:23.281689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6c9b4-ccd5-47b8-bac4-28a79b8bbf05 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:23.305401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1315/2607] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:23 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:23.309417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9f86b-ee9d-4fcf-b5cb-c93bc8d6a2c1 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:23.320825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9f86b-ee9d-4fcf-b5cb-c93bc8d6a2c1 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:23.325796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51184ce6-ce45-4ba6-8939-4ba308acb664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:23.326475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51184ce6-ce45-4ba6-8939-4ba308acb664 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:23.326784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51184ce6-ce45-4ba6-8939-4ba308acb664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:23.327060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51184ce6-ce45-4ba6-8939-4ba308acb664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:23.327717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51184ce6-ce45-4ba6-8939-4ba308acb664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:23.329371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51184ce6-ce45-4ba6-8939-4ba308acb664 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:23.341404 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1293/2608] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:23.345549 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6c9b4-ccd5-47b8-bac4-28a79b8bbf05 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:23.356352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6c9b4-ccd5-47b8-bac4-28a79b8bbf05 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:23.361220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87382524-e96c-425e-9441-be3df9b70a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:23.361875 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87382524-e96c-425e-9441-be3df9b70a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:23.362153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87382524-e96c-425e-9441-be3df9b70a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:23.363005 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87382524-e96c-425e-9441-be3df9b70a96 admin admin] PATH_INFO: `/v3/projects/f62014b9d5b9457ca4e1e6b1d1da0919/users/ec17b7d0b9764ab1bc198249c3ffe571/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:23.378797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87382524-e96c-425e-9441-be3df9b70a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f62014b9d5b9457ca4e1e6b1d1da0919, user_id=ec17b7d0b9764ab1bc198249c3ffe571, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:23.381489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87382524-e96c-425e-9441-be3df9b70a96 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:23.395675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1316/2609] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:23 2022] PUT /identity/v3/projects/f62014b9d5b9457ca4e1e6b1d1da0919/users/ec17b7d0b9764ab1bc198249c3ffe571/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:23.399744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51184ce6-ce45-4ba6-8939-4ba308acb664 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:23.411610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51184ce6-ce45-4ba6-8939-4ba308acb664 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:23.483566 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00029241-9b96-4a3a-87f1-2d51dff5d76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:23.484311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00029241-9b96-4a3a-87f1-2d51dff5d76b admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:23.484687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00029241-9b96-4a3a-87f1-2d51dff5d76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:23.485282 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00029241-9b96-4a3a-87f1-2d51dff5d76b admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:23.486011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00029241-9b96-4a3a-87f1-2d51dff5d76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:23.488341 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00029241-9b96-4a3a-87f1-2d51dff5d76b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:23.502779 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1294/2610] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:23.506832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87382524-e96c-425e-9441-be3df9b70a96 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:23.522307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87382524-e96c-425e-9441-be3df9b70a96 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:23.529446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764194e9-2d38-481c-bfd4-780f17b7bab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:23.530125 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764194e9-2d38-481c-bfd4-780f17b7bab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:23.530400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764194e9-2d38-481c-bfd4-780f17b7bab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:23.530661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764194e9-2d38-481c-bfd4-780f17b7bab0 admin admin] PATH_INFO: `/v3/projects/f62014b9d5b9457ca4e1e6b1d1da0919/users/ec17b7d0b9764ab1bc198249c3ffe571/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:23.532758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764194e9-2d38-481c-bfd4-780f17b7bab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f62014b9d5b9457ca4e1e6b1d1da0919, user_id=ec17b7d0b9764ab1bc198249c3ffe571, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:23.536199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764194e9-2d38-481c-bfd4-780f17b7bab0 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:23.553178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1317/2611] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:23 2022] PUT /identity/v3/projects/f62014b9d5b9457ca4e1e6b1d1da0919/users/ec17b7d0b9764ab1bc198249c3ffe571/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:24.449273 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00029241-9b96-4a3a-87f1-2d51dff5d76b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:24.505407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:24.536904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00029241-9b96-4a3a-87f1-2d51dff5d76b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:24.617386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fd521-88d0-41e7-ac62-7063b2937b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:24.618014 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fd521-88d0-41e7-ac62-7063b2937b15 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:24.618298 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fd521-88d0-41e7-ac62-7063b2937b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:24.618572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fd521-88d0-41e7-ac62-7063b2937b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:24.620786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ed1fd521-88d0-41e7-ac62-7063b2937b15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:24.705009 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fd521-88d0-41e7-ac62-7063b2937b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:24.707146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fd521-88d0-41e7-ac62-7063b2937b15 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:24.717851 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1295/2612] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:29.944963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8fa9e-7e56-4012-bb0b-a2b745a89924 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:29.945531 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8fa9e-7e56-4012-bb0b-a2b745a89924 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:29.945531 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8fa9e-7e56-4012-bb0b-a2b745a89924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:29.977198 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-4dc8fa9e-7e56-4012-bb0b-a2b745a89924 None None] MFA Rules not processed for user `ec17b7d0b9764ab1bc198249c3ffe571`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:29.978451 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4dc8fa9e-7e56-4012-bb0b-a2b745a89924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:30.093741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1318/2613] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:30.110839 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fd521-88d0-41e7-ac62-7063b2937b15 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:30.112308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ed1fd521-88d0-41e7-ac62-7063b2937b15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:30.134143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8fa9e-7e56-4012-bb0b-a2b745a89924 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:30.135604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4dc8fa9e-7e56-4012-bb0b-a2b745a89924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:30.152494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8fa9e-7e56-4012-bb0b-a2b745a89924 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:30.158275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:30.158982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:30.159829 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:30.160117 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] PATH_INFO: `/v3/users/b0cbf49cfca74838bee4c8162dfe27ef` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:30.161480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0cbf49cfca74838bee4c8162dfe27ef)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:30.163648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:30.178344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:30.208433 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fd521-88d0-41e7-ac62-7063b2937b15 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:30.215604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d403b15-13cd-4bde-bb48-176d32157f04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:30.216230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d403b15-13cd-4bde-bb48-176d32157f04 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:30.216507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d403b15-13cd-4bde-bb48-176d32157f04 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:30.216945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d403b15-13cd-4bde-bb48-176d32157f04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:30.248415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0cbf49cfca74838bee4c8162dfe27ef'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:30.264967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:30.280066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0cbf49cfca74838bee4c8162dfe27ef'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:30.281278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0cbf49cfca74838bee4c8162dfe27ef'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:30.291356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0cbf49cfca74838bee4c8162dfe27ef'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:30.292115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0cbf49cfca74838bee4c8162dfe27ef'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:30.302952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d403b15-13cd-4bde-bb48-176d32157f04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:30.305034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d403b15-13cd-4bde-bb48-176d32157f04 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:30.318651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1319/2614] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:30 2022] DELETE /identity/v3/users/b0cbf49cfca74838bee4c8162dfe27ef => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:30.321481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1296/2615] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:50:30.550139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:30.552772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:30.668552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf13d7f-917f-4128-82a4-46d9a47bcf7a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:30.700256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1108ddc-042b-4eea-b8f4-91975f10d391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:30.701568 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1108ddc-042b-4eea-b8f4-91975f10d391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:30.701953 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1108ddc-042b-4eea-b8f4-91975f10d391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:30.702240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1108ddc-042b-4eea-b8f4-91975f10d391 admin admin] PATH_INFO: `/v3/projects/c067f1df3c7f47299bc2db03d904d156` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:30.703471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1108ddc-042b-4eea-b8f4-91975f10d391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c067f1df3c7f47299bc2db03d904d156)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:30.705535 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1108ddc-042b-4eea-b8f4-91975f10d391 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:30.778775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-a1108ddc-042b-4eea-b8f4-91975f10d391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c067f1df3c7f47299bc2db03d904d156', 'request_id': 'req-a1108ddc-042b-4eea-b8f4-91975f10d391'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:30.786111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-a1108ddc-042b-4eea-b8f4-91975f10d391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c067f1df3c7f47299bc2db03d904d156', 'request_id': 'req-a1108ddc-042b-4eea-b8f4-91975f10d391'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:30.793534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-a1108ddc-042b-4eea-b8f4-91975f10d391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c067f1df3c7f47299bc2db03d904d156', 'request_id': 'req-a1108ddc-042b-4eea-b8f4-91975f10d391'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:30.808990 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-a1108ddc-042b-4eea-b8f4-91975f10d391 admin admin] The token cache is being invalidate because project c067f1df3c7f47299bc2db03d904d156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:30.809428 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-a1108ddc-042b-4eea-b8f4-91975f10d391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:30.810146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-a1108ddc-042b-4eea-b8f4-91975f10d391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:30.812138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1320/2616] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:30 2022] DELETE /identity/v3/projects/c067f1df3c7f47299bc2db03d904d156 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:30.817284 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777d3f6-878b-4327-88d3-7f26ef954f07 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:30.817578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777d3f6-878b-4327-88d3-7f26ef954f07 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:30.817855 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777d3f6-878b-4327-88d3-7f26ef954f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:30.844322 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-c777d3f6-878b-4327-88d3-7f26ef954f07 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:30.846043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c777d3f6-878b-4327-88d3-7f26ef954f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:30.925738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1297/2617] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:30.934769 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdfbec-ae88-4eb8-ba15-4e174b0ef4d7 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:30.935055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdfbec-ae88-4eb8-ba15-4e174b0ef4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:30.935255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdfbec-ae88-4eb8-ba15-4e174b0ef4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:30.958153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5cbdfbec-ae88-4eb8-ba15-4e174b0ef4d7 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:30.959501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5cbdfbec-ae88-4eb8-ba15-4e174b0ef4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:30.968351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1321/2618] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:30.973276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777d3f6-878b-4327-88d3-7f26ef954f07 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:30.983830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777d3f6-878b-4327-88d3-7f26ef954f07 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:30.991242 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564687bd-6937-4aa3-846a-4143be43aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:30.992091 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564687bd-6937-4aa3-846a-4143be43aaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:30.992558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564687bd-6937-4aa3-846a-4143be43aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:30.993018 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564687bd-6937-4aa3-846a-4143be43aaa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:30.993870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564687bd-6937-4aa3-846a-4143be43aaa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:50:30.994343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564687bd-6937-4aa3-846a-4143be43aaa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:30.995974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564687bd-6937-4aa3-846a-4143be43aaa6 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:31.010831 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1298/2619] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:50:31.015580 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdfbec-ae88-4eb8-ba15-4e174b0ef4d7 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:31.026873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdfbec-ae88-4eb8-ba15-4e174b0ef4d7 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:31.032649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321374a-86dc-482a-b92f-0c4f9d12ffff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:31.034320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321374a-86dc-482a-b92f-0c4f9d12ffff admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:31.034717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321374a-86dc-482a-b92f-0c4f9d12ffff admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:31.035087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321374a-86dc-482a-b92f-0c4f9d12ffff admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:31.035819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321374a-86dc-482a-b92f-0c4f9d12ffff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:31.038236 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321374a-86dc-482a-b92f-0c4f9d12ffff admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:31.050248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:31.074300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1322/2620] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:31 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:31.078538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564687bd-6937-4aa3-846a-4143be43aaa6 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:31.089644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564687bd-6937-4aa3-846a-4143be43aaa6 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:31.157344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25355300-a643-4991-81b2-f9b512193d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:31.158752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25355300-a643-4991-81b2-f9b512193d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:31.159615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25355300-a643-4991-81b2-f9b512193d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:31.159950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25355300-a643-4991-81b2-f9b512193d80 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:31.160820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25355300-a643-4991-81b2-f9b512193d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:31.162926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25355300-a643-4991-81b2-f9b512193d80 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:31.187997 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1299/2621] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:31 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:31.192150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321374a-86dc-482a-b92f-0c4f9d12ffff admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:31.202845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321374a-86dc-482a-b92f-0c4f9d12ffff admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:31.208395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e3a6a-bcfc-4627-ab97-f18d5165f4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:31.209145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3a6a-bcfc-4627-ab97-f18d5165f4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:31.209538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3a6a-bcfc-4627-ab97-f18d5165f4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:31.209909 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3a6a-bcfc-4627-ab97-f18d5165f4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:31.210654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e3a6a-bcfc-4627-ab97-f18d5165f4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:31.212932 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e3a6a-bcfc-4627-ab97-f18d5165f4a4 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:31.230996 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1323/2622] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:31.231631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25355300-a643-4991-81b2-f9b512193d80 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:31.247031 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25355300-a643-4991-81b2-f9b512193d80 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:31.253960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabf8e9-f360-4962-877b-159c99eae7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:31.254827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabf8e9-f360-4962-877b-159c99eae7cd admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:31.255139 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabf8e9-f360-4962-877b-159c99eae7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:31.255716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabf8e9-f360-4962-877b-159c99eae7cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:31.256815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabf8e9-f360-4962-877b-159c99eae7cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:31.259007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabf8e9-f360-4962-877b-159c99eae7cd admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:31.276326 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1300/2623] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:31.279921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e3a6a-bcfc-4627-ab97-f18d5165f4a4 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:31.292177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e3a6a-bcfc-4627-ab97-f18d5165f4a4 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:31.298068 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee62a6-2cf5-4e69-b396-1cdc340673ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:31.298818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee62a6-2cf5-4e69-b396-1cdc340673ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:31.299169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee62a6-2cf5-4e69-b396-1cdc340673ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:31.299561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee62a6-2cf5-4e69-b396-1cdc340673ec admin admin] PATH_INFO: `/v3/projects/8894ac55d9fa46c1bbcc71e8140a8673/users/3a3af249f1254656a55b66c201af883f/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:31.320738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee62a6-2cf5-4e69-b396-1cdc340673ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=8894ac55d9fa46c1bbcc71e8140a8673, user_id=3a3af249f1254656a55b66c201af883f, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:31.324508 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee62a6-2cf5-4e69-b396-1cdc340673ec admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:31.339506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1324/2624] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:31 2022] PUT /identity/v3/projects/8894ac55d9fa46c1bbcc71e8140a8673/users/3a3af249f1254656a55b66c201af883f/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:31.350261 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693dcc2-f4c0-412c-a473-f6525efd3bcf None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:31.350555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693dcc2-f4c0-412c-a473-f6525efd3bcf None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:31.350830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693dcc2-f4c0-412c-a473-f6525efd3bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:31.382334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-f693dcc2-f4c0-412c-a473-f6525efd3bcf None None] MFA Rules not processed for user `3a3af249f1254656a55b66c201af883f`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:31.383999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-f693dcc2-f4c0-412c-a473-f6525efd3bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:31.428909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:31.472710 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1301/2625] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:31.477477 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee62a6-2cf5-4e69-b396-1cdc340673ec admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:31.545273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee62a6-2cf5-4e69-b396-1cdc340673ec admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:31.550217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1d8af-98d8-4ad9-abe7-a496a37240ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:31.550833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1d8af-98d8-4ad9-abe7-a496a37240ab admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:31.551107 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1d8af-98d8-4ad9-abe7-a496a37240ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:31.551385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1d8af-98d8-4ad9-abe7-a496a37240ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:31.552034 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1d8af-98d8-4ad9-abe7-a496a37240ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:31.554093 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1d8af-98d8-4ad9-abe7-a496a37240ab admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:31.589149 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1325/2626] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:31 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:31.593425 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693dcc2-f4c0-412c-a473-f6525efd3bcf None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:31.606246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693dcc2-f4c0-412c-a473-f6525efd3bcf None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:31.680303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac5c35-5395-40a4-8f52-2223c5206a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:31.680991 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac5c35-5395-40a4-8f52-2223c5206a9f admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:31.681440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac5c35-5395-40a4-8f52-2223c5206a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:31.681781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac5c35-5395-40a4-8f52-2223c5206a9f admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:31.683713 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac5c35-5395-40a4-8f52-2223c5206a9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:31.684802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac5c35-5395-40a4-8f52-2223c5206a9f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:31.716255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1302/2627] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:31 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:31.720169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1d8af-98d8-4ad9-abe7-a496a37240ab admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:31.731427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1d8af-98d8-4ad9-abe7-a496a37240ab admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:31.738772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823496d-3362-45a1-bec6-d6c0fe62140c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:31.739461 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823496d-3362-45a1-bec6-d6c0fe62140c admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:31.739808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823496d-3362-45a1-bec6-d6c0fe62140c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:31.740175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823496d-3362-45a1-bec6-d6c0fe62140c admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:31.740962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823496d-3362-45a1-bec6-d6c0fe62140c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:31.742653 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823496d-3362-45a1-bec6-d6c0fe62140c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:31.755361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1326/2628] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:31.759646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac5c35-5395-40a4-8f52-2223c5206a9f admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:31.770730 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac5c35-5395-40a4-8f52-2223c5206a9f admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:31.775868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49297b9-fd8a-4bb1-9492-06e616eb92ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:31.776707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49297b9-fd8a-4bb1-9492-06e616eb92ed admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:31.777056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49297b9-fd8a-4bb1-9492-06e616eb92ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:31.777426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49297b9-fd8a-4bb1-9492-06e616eb92ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:31.778142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49297b9-fd8a-4bb1-9492-06e616eb92ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:31.779797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49297b9-fd8a-4bb1-9492-06e616eb92ed admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:31.791610 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1303/2629] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:31.797262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823496d-3362-45a1-bec6-d6c0fe62140c admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:31.808038 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823496d-3362-45a1-bec6-d6c0fe62140c admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:31.813209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e4b69-a4b3-4428-8590-84088113e168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:31.813882 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e4b69-a4b3-4428-8590-84088113e168 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:31.814160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e4b69-a4b3-4428-8590-84088113e168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:31.814437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e4b69-a4b3-4428-8590-84088113e168 admin admin] PATH_INFO: `/v3/projects/672edf23719e45aaa4407dc57fea2cc9/users/93e18abf16a84222b741e2dad70a16fb/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:31.831337 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e4b69-a4b3-4428-8590-84088113e168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=672edf23719e45aaa4407dc57fea2cc9, user_id=93e18abf16a84222b741e2dad70a16fb, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:31.834061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e4b69-a4b3-4428-8590-84088113e168 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:31.854902 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1327/2630] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:31 2022] PUT /identity/v3/projects/672edf23719e45aaa4407dc57fea2cc9/users/93e18abf16a84222b741e2dad70a16fb/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:31.865387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44da648-9b5f-4125-8899-09fd12ed89fe None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:31.865697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44da648-9b5f-4125-8899-09fd12ed89fe None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:31.866363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44da648-9b5f-4125-8899-09fd12ed89fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:31.899068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-b44da648-9b5f-4125-8899-09fd12ed89fe None None] MFA Rules not processed for user `93e18abf16a84222b741e2dad70a16fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:31.900754 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b44da648-9b5f-4125-8899-09fd12ed89fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:31.989961 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1304/2631] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:31.993349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e4b69-a4b3-4428-8590-84088113e168 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:32.073351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e4b69-a4b3-4428-8590-84088113e168 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:32.079729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eebaf-9326-4a81-95dc-0766ef34f003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:32.080309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eebaf-9326-4a81-95dc-0766ef34f003 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:32.081167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eebaf-9326-4a81-95dc-0766ef34f003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:32.081509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eebaf-9326-4a81-95dc-0766ef34f003 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:32.082110 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eebaf-9326-4a81-95dc-0766ef34f003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:32.084537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eebaf-9326-4a81-95dc-0766ef34f003 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:32.095315 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:32.119494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1328/2632] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:31 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:32.123741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44da648-9b5f-4125-8899-09fd12ed89fe None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:32.139196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44da648-9b5f-4125-8899-09fd12ed89fe None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:32.182515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:32.202306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36d86f-9025-49c2-9fb1-61ea05c9828a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:32.202920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36d86f-9025-49c2-9fb1-61ea05c9828a admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:32.203207 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36d86f-9025-49c2-9fb1-61ea05c9828a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:32.203479 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36d86f-9025-49c2-9fb1-61ea05c9828a admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:32.204125 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36d86f-9025-49c2-9fb1-61ea05c9828a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:32.206797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36d86f-9025-49c2-9fb1-61ea05c9828a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:32.228047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1305/2633] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:32 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:50:32.231687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eebaf-9326-4a81-95dc-0766ef34f003 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:32.241840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eebaf-9326-4a81-95dc-0766ef34f003 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:32.246783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a95ddc-f8b9-4cf5-919b-e2cb670d31e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:32.247384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a95ddc-f8b9-4cf5-919b-e2cb670d31e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:32.247664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a95ddc-f8b9-4cf5-919b-e2cb670d31e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:32.247934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a95ddc-f8b9-4cf5-919b-e2cb670d31e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:32.248584 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a95ddc-f8b9-4cf5-919b-e2cb670d31e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:32.250273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a95ddc-f8b9-4cf5-919b-e2cb670d31e0 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:32.261503 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1329/2634] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:32.265769 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36d86f-9025-49c2-9fb1-61ea05c9828a admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:32.276554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36d86f-9025-49c2-9fb1-61ea05c9828a admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:32.281141 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39244937-58b6-4614-ba51-bdd902cba852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:32.281794 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39244937-58b6-4614-ba51-bdd902cba852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:32.282072 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39244937-58b6-4614-ba51-bdd902cba852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:32.282343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39244937-58b6-4614-ba51-bdd902cba852 admin admin] PATH_INFO: `/v3/projects/38ca8a6b996840b4835535742887ff8f/users/fb8ad2f026454622832607c24c1d9da6/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:32.297422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39244937-58b6-4614-ba51-bdd902cba852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ca8a6b996840b4835535742887ff8f, user_id=fb8ad2f026454622832607c24c1d9da6, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:32.300104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39244937-58b6-4614-ba51-bdd902cba852 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:32.313974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1306/2635] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:32 2022] PUT /identity/v3/projects/38ca8a6b996840b4835535742887ff8f/users/fb8ad2f026454622832607c24c1d9da6/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:32.318919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a95ddc-f8b9-4cf5-919b-e2cb670d31e0 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:32.328826 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a95ddc-f8b9-4cf5-919b-e2cb670d31e0 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:32.385052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1a9ac-3584-40de-aafb-fb6000fed3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:32.385687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1a9ac-3584-40de-aafb-fb6000fed3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:32.386160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1a9ac-3584-40de-aafb-fb6000fed3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:32.386301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1a9ac-3584-40de-aafb-fb6000fed3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:32.386892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1a9ac-3584-40de-aafb-fb6000fed3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:32.388652 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1a9ac-3584-40de-aafb-fb6000fed3a4 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:32.405414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1330/2636] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:32.406491 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39244937-58b6-4614-ba51-bdd902cba852 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:32.416732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39244937-58b6-4614-ba51-bdd902cba852 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:32.421626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108faa59-887d-4352-b88e-99dde7850d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:32.422268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108faa59-887d-4352-b88e-99dde7850d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:32.422544 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108faa59-887d-4352-b88e-99dde7850d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:32.422813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108faa59-887d-4352-b88e-99dde7850d25 admin admin] PATH_INFO: `/v3/projects/38ca8a6b996840b4835535742887ff8f/users/fb8ad2f026454622832607c24c1d9da6/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:32.424725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108faa59-887d-4352-b88e-99dde7850d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ca8a6b996840b4835535742887ff8f, user_id=fb8ad2f026454622832607c24c1d9da6, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:32.427366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108faa59-887d-4352-b88e-99dde7850d25 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:32.437943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1307/2637] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:32 2022] PUT /identity/v3/projects/38ca8a6b996840b4835535742887ff8f/users/fb8ad2f026454622832607c24c1d9da6/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:32.449980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdaed9-4c0c-4bb1-8781-e3a6910fd415 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:32.450264 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdaed9-4c0c-4bb1-8781-e3a6910fd415 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:32.450538 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdaed9-4c0c-4bb1-8781-e3a6910fd415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:32.482642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-c9fdaed9-4c0c-4bb1-8781-e3a6910fd415 None None] MFA Rules not processed for user `fb8ad2f026454622832607c24c1d9da6`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:32.483880 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c9fdaed9-4c0c-4bb1-8781-e3a6910fd415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:32.553215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1331/2638] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:32.564646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108faa59-887d-4352-b88e-99dde7850d25 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:32.566956 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-108faa59-887d-4352-b88e-99dde7850d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:32.626196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108faa59-887d-4352-b88e-99dde7850d25 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:32.630598 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8570d28-6d74-42dc-a8b3-e1e61d9eabe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:32.631232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8570d28-6d74-42dc-a8b3-e1e61d9eabe0 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:32.631504 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8570d28-6d74-42dc-a8b3-e1e61d9eabe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:32.631773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8570d28-6d74-42dc-a8b3-e1e61d9eabe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:32.644129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8570d28-6d74-42dc-a8b3-e1e61d9eabe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:32.645574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8570d28-6d74-42dc-a8b3-e1e61d9eabe0 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:32.655008 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1308/2639] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:33.163851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdaed9-4c0c-4bb1-8781-e3a6910fd415 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:33.173713 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdaed9-4c0c-4bb1-8781-e3a6910fd415 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:33.178117 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc0e2d-3875-4fc9-99d9-98d226008ada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:33.178794 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc0e2d-3875-4fc9-99d9-98d226008ada service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:33.179081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc0e2d-3875-4fc9-99d9-98d226008ada service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:33.179357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc0e2d-3875-4fc9-99d9-98d226008ada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:33.236144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc0e2d-3875-4fc9-99d9-98d226008ada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:33.237622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc0e2d-3875-4fc9-99d9-98d226008ada service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:33.241100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:33.247955 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1332/2640] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:39.488950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8570d28-6d74-42dc-a8b3-e1e61d9eabe0 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:39.497940 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8570d28-6d74-42dc-a8b3-e1e61d9eabe0 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:39.501801 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521d83b-6657-4577-a45f-a438469f3c24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:39.503057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521d83b-6657-4577-a45f-a438469f3c24 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:39.503363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521d83b-6657-4577-a45f-a438469f3c24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:39.503613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521d83b-6657-4577-a45f-a438469f3c24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:39.576519 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521d83b-6657-4577-a45f-a438469f3c24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:39.578019 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521d83b-6657-4577-a45f-a438469f3c24 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:39.591349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc0e2d-3875-4fc9-99d9-98d226008ada service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:39.592968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-33dc0e2d-3875-4fc9-99d9-98d226008ada service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:39.596293 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1309/2641] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:39.673680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc0e2d-3875-4fc9-99d9-98d226008ada service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:39.682923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:39.682923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:39.682923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:39.682923 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] PATH_INFO: `/v3/users/377ff79677314498a2b7b1500ef2f069` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:39.683443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=377ff79677314498a2b7b1500ef2f069)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:39.685290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:39.747546 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377ff79677314498a2b7b1500ef2f069'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:39.775617 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377ff79677314498a2b7b1500ef2f069'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:39.776679 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377ff79677314498a2b7b1500ef2f069'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:39.785466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377ff79677314498a2b7b1500ef2f069'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:39.786187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377ff79677314498a2b7b1500ef2f069'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:39.803939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1333/2642] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:39 2022] DELETE /identity/v3/users/377ff79677314498a2b7b1500ef2f069 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:39.808455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54969534-4534-4464-aa7b-6581aab60af6 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:39.808741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54969534-4534-4464-aa7b-6581aab60af6 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:39.809017 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54969534-4534-4464-aa7b-6581aab60af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:39.838380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-54969534-4534-4464-aa7b-6581aab60af6 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:39.841159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-54969534-4534-4464-aa7b-6581aab60af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:39.877874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:39.913654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1310/2643] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:39.921646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:39.923087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:39.988572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d16515-c50e-42ac-9a03-4f199be87d7a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:39.993400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28b0d5-c3a7-4a16-9f50-c9f52c37f1b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:39.994017 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28b0d5-c3a7-4a16-9f50-c9f52c37f1b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:39.994933 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28b0d5-c3a7-4a16-9f50-c9f52c37f1b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:39.995212 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28b0d5-c3a7-4a16-9f50-c9f52c37f1b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:40.008872 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28b0d5-c3a7-4a16-9f50-c9f52c37f1b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:40.010812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28b0d5-c3a7-4a16-9f50-c9f52c37f1b7 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:40.021397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1334/2644] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:40.602495 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54969534-4534-4464-aa7b-6581aab60af6 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:40.604038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-54969534-4534-4464-aa7b-6581aab60af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:40.617912 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54969534-4534-4464-aa7b-6581aab60af6 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:40.625251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:40.627022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:40.627022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:40.627022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] PATH_INFO: `/v3/projects/63bb9bb9921041d49d13658fb2218f2e` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:40.628445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=63bb9bb9921041d49d13658fb2218f2e)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:40.629784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:40.679734 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63bb9bb9921041d49d13658fb2218f2e', 'request_id': 'req-a1ca643c-d014-4630-bf69-3aac766fad3f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:40.685701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63bb9bb9921041d49d13658fb2218f2e', 'request_id': 'req-a1ca643c-d014-4630-bf69-3aac766fad3f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:40.693394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63bb9bb9921041d49d13658fb2218f2e', 'request_id': 'req-a1ca643c-d014-4630-bf69-3aac766fad3f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:40.704559 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] The token cache is being invalidate because project 63bb9bb9921041d49d13658fb2218f2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:40.704846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:40.705511 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:40.707027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1311/2645] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:40 2022] DELETE /identity/v3/projects/63bb9bb9921041d49d13658fb2218f2e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:40.711768 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28b0d5-c3a7-4a16-9f50-c9f52c37f1b7 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:40.722256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28b0d5-c3a7-4a16-9f50-c9f52c37f1b7 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:40.740383 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:40.785413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23393a0d-d62a-43ef-bc54-d2d600a6c119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:40.786046 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23393a0d-d62a-43ef-bc54-d2d600a6c119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:40.786331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23393a0d-d62a-43ef-bc54-d2d600a6c119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:40.786609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23393a0d-d62a-43ef-bc54-d2d600a6c119 admin admin] PATH_INFO: `/v3/users/1a48727e15e64f36a0a8d02cf0e42a0d` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:40.787734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23393a0d-d62a-43ef-bc54-d2d600a6c119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a48727e15e64f36a0a8d02cf0e42a0d)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:40.789745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23393a0d-d62a-43ef-bc54-d2d600a6c119 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:40.840188 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-23393a0d-d62a-43ef-bc54-d2d600a6c119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a48727e15e64f36a0a8d02cf0e42a0d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:40.853261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-23393a0d-d62a-43ef-bc54-d2d600a6c119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a48727e15e64f36a0a8d02cf0e42a0d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:40.853963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-23393a0d-d62a-43ef-bc54-d2d600a6c119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a48727e15e64f36a0a8d02cf0e42a0d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:40.861081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-23393a0d-d62a-43ef-bc54-d2d600a6c119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a48727e15e64f36a0a8d02cf0e42a0d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:40.861796 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-23393a0d-d62a-43ef-bc54-d2d600a6c119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a48727e15e64f36a0a8d02cf0e42a0d'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:40.875211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1335/2646] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:40 2022] DELETE /identity/v3/users/1a48727e15e64f36a0a8d02cf0e42a0d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:41.167476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:41.169072 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:41.190432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca643c-d014-4630-bf69-3aac766fad3f admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:41.248950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:41.249655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:41.249939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:41.250291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4 admin admin] PATH_INFO: `/v3/projects/cb6ae49ff8cf4cdc8d58ce43523d0558` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:41.251422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6ae49ff8cf4cdc8d58ce43523d0558)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:41.254001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:41.303466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6ae49ff8cf4cdc8d58ce43523d0558', 'request_id': 'req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:41.309173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6ae49ff8cf4cdc8d58ce43523d0558', 'request_id': 'req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:41.310874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:41.318221 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6ae49ff8cf4cdc8d58ce43523d0558', 'request_id': 'req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:41.330192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4 admin admin] The token cache is being invalidate because project cb6ae49ff8cf4cdc8d58ce43523d0558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:41.331020 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:41.331674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b2bfdaba-7cc1-4db4-962e-a09e89a956e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:41.333502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1312/2647] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:41 2022] DELETE /identity/v3/projects/cb6ae49ff8cf4cdc8d58ce43523d0558 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:41.338576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10856b-cc8e-48a8-9dcf-8dfec12b4396 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:41.338863 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10856b-cc8e-48a8-9dcf-8dfec12b4396 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:41.339148 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10856b-cc8e-48a8-9dcf-8dfec12b4396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:41.358466 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-0d10856b-cc8e-48a8-9dcf-8dfec12b4396 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:41.359667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-0d10856b-cc8e-48a8-9dcf-8dfec12b4396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:41.426183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1336/2648] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:41.434834 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343418ac-b059-4b77-83f5-e41d33bba77f None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:41.435121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343418ac-b059-4b77-83f5-e41d33bba77f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:41.435401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343418ac-b059-4b77-83f5-e41d33bba77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:41.456883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-343418ac-b059-4b77-83f5-e41d33bba77f None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:41.457963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-343418ac-b059-4b77-83f5-e41d33bba77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:41.468294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1313/2649] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:41.472917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10856b-cc8e-48a8-9dcf-8dfec12b4396 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:41.483286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10856b-cc8e-48a8-9dcf-8dfec12b4396 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:41.487962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db088e-8bba-487c-9ed5-a842dc654dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:41.488570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db088e-8bba-487c-9ed5-a842dc654dae admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:41.488854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db088e-8bba-487c-9ed5-a842dc654dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:41.489154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db088e-8bba-487c-9ed5-a842dc654dae admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:41.489805 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db088e-8bba-487c-9ed5-a842dc654dae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:50:41.490111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db088e-8bba-487c-9ed5-a842dc654dae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:41.491673 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db088e-8bba-487c-9ed5-a842dc654dae admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:41.505137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1337/2650] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:50:41.509399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343418ac-b059-4b77-83f5-e41d33bba77f None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:41.531320 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343418ac-b059-4b77-83f5-e41d33bba77f None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:41.536176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de5941-1938-49a0-af5d-5eeca21a9c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:41.536817 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de5941-1938-49a0-af5d-5eeca21a9c99 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:41.537116 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de5941-1938-49a0-af5d-5eeca21a9c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:41.537401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de5941-1938-49a0-af5d-5eeca21a9c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:41.538034 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de5941-1938-49a0-af5d-5eeca21a9c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:41.539945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de5941-1938-49a0-af5d-5eeca21a9c99 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:41.571920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1314/2651] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:41 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:41.576519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db088e-8bba-487c-9ed5-a842dc654dae admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:41.588334 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db088e-8bba-487c-9ed5-a842dc654dae admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:41.617567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:41.665255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f631ce9-3a1c-4e19-b3d0-78cd23715efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:41.665945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f631ce9-3a1c-4e19-b3d0-78cd23715efd admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:41.666220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f631ce9-3a1c-4e19-b3d0-78cd23715efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:41.666498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f631ce9-3a1c-4e19-b3d0-78cd23715efd admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:41.667128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f631ce9-3a1c-4e19-b3d0-78cd23715efd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:41.669114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f631ce9-3a1c-4e19-b3d0-78cd23715efd admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:41.716182 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1338/2652] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:41 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:41.718289 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de5941-1938-49a0-af5d-5eeca21a9c99 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:41.731967 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de5941-1938-49a0-af5d-5eeca21a9c99 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:41.739542 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1df62-009c-4553-87b5-2f0d5e575dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:41.740113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1df62-009c-4553-87b5-2f0d5e575dcc admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:41.740395 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1df62-009c-4553-87b5-2f0d5e575dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:41.740674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1df62-009c-4553-87b5-2f0d5e575dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:41.741361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1df62-009c-4553-87b5-2f0d5e575dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:41.742956 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1df62-009c-4553-87b5-2f0d5e575dcc admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:41.754164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1315/2653] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:41.758600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f631ce9-3a1c-4e19-b3d0-78cd23715efd admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:41.770298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f631ce9-3a1c-4e19-b3d0-78cd23715efd admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:41.775522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13a491-7363-4ca0-a475-4da72d4cb5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:41.776154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a491-7363-4ca0-a475-4da72d4cb5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:41.776431 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a491-7363-4ca0-a475-4da72d4cb5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:41.776703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a491-7363-4ca0-a475-4da72d4cb5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:41.777504 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13a491-7363-4ca0-a475-4da72d4cb5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:41.779353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13a491-7363-4ca0-a475-4da72d4cb5a4 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:41.795509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1339/2654] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:41.796911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1df62-009c-4553-87b5-2f0d5e575dcc admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:41.810424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1df62-009c-4553-87b5-2f0d5e575dcc admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:41.815269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efd19f-4341-4cd9-9907-4008e8990ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:41.815926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efd19f-4341-4cd9-9907-4008e8990ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:41.816211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efd19f-4341-4cd9-9907-4008e8990ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:41.816490 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efd19f-4341-4cd9-9907-4008e8990ba0 admin admin] PATH_INFO: `/v3/projects/e13b964ffdf045f7be92eadd25e35a28/users/ab4d7e2204074c78bf118eba7c9b3d6a/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:41.839589 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efd19f-4341-4cd9-9907-4008e8990ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13b964ffdf045f7be92eadd25e35a28, user_id=ab4d7e2204074c78bf118eba7c9b3d6a, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:41.842634 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efd19f-4341-4cd9-9907-4008e8990ba0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:41.862618 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1316/2655] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:41 2022] PUT /identity/v3/projects/e13b964ffdf045f7be92eadd25e35a28/users/ab4d7e2204074c78bf118eba7c9b3d6a/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:41.873442 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faddd4-8dff-4827-a414-a37dbc30d847 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:41.873764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faddd4-8dff-4827-a414-a37dbc30d847 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:41.873888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efd19f-4341-4cd9-9907-4008e8990ba0 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:41.874032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faddd4-8dff-4827-a414-a37dbc30d847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:41.875612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-83efd19f-4341-4cd9-9907-4008e8990ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:41.910623 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-19faddd4-8dff-4827-a414-a37dbc30d847 None None] MFA Rules not processed for user `ab4d7e2204074c78bf118eba7c9b3d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:41.911790 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-19faddd4-8dff-4827-a414-a37dbc30d847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:41.956731 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efd19f-4341-4cd9-9907-4008e8990ba0 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:41.965868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1715b76-1c70-4e23-938f-238b7e24515b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:41.967397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1715b76-1c70-4e23-938f-238b7e24515b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:41.968041 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1715b76-1c70-4e23-938f-238b7e24515b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:41.968889 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1715b76-1c70-4e23-938f-238b7e24515b admin admin] PATH_INFO: `/v3/users/3a3af249f1254656a55b66c201af883f` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:41.970811 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1715b76-1c70-4e23-938f-238b7e24515b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3af249f1254656a55b66c201af883f)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:41.973365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1715b76-1c70-4e23-938f-238b7e24515b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:42.008903 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1340/2656] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:42.013758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19faddd4-8dff-4827-a414-a37dbc30d847 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:42.026543 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19faddd4-8dff-4827-a414-a37dbc30d847 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:42.033312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0199af0-80a2-447c-9cba-059e1a8cf85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:42.034225 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0199af0-80a2-447c-9cba-059e1a8cf85b admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.034512 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0199af0-80a2-447c-9cba-059e1a8cf85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.034794 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0199af0-80a2-447c-9cba-059e1a8cf85b admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.035448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0199af0-80a2-447c-9cba-059e1a8cf85b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:42.037472 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0199af0-80a2-447c-9cba-059e1a8cf85b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:42.045922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e1715b76-1c70-4e23-938f-238b7e24515b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3af249f1254656a55b66c201af883f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.054046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e1715b76-1c70-4e23-938f-238b7e24515b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3af249f1254656a55b66c201af883f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.054822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e1715b76-1c70-4e23-938f-238b7e24515b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3af249f1254656a55b66c201af883f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.055444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e1715b76-1c70-4e23-938f-238b7e24515b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3af249f1254656a55b66c201af883f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.064558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e1715b76-1c70-4e23-938f-238b7e24515b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3af249f1254656a55b66c201af883f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.071885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1341/2657] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:42 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:50:42.075886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0199af0-80a2-447c-9cba-059e1a8cf85b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:42.077520 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a0199af0-80a2-447c-9cba-059e1a8cf85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:42.086853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1317/2658] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:41 2022] DELETE /identity/v3/users/3a3af249f1254656a55b66c201af883f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:42.091390 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1a6ce-62c4-4e02-8f2b-a9f5a041ad4b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.091726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1a6ce-62c4-4e02-8f2b-a9f5a041ad4b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.091982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1a6ce-62c4-4e02-8f2b-a9f5a041ad4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.099208 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:INSERT=4,keystone:DELETE=11,keystone:SELECT=85 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:42.115663 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-eab1a6ce-62c4-4e02-8f2b-a9f5a041ad4b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:42.116900 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-eab1a6ce-62c4-4e02-8f2b-a9f5a041ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:42.152246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0199af0-80a2-447c-9cba-059e1a8cf85b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:42.176908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:42.184882 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1318/2659] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:42.195786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1a6ce-62c4-4e02-8f2b-a9f5a041ad4b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:42.197216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-eab1a6ce-62c4-4e02-8f2b-a9f5a041ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:42.218820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a2b3c-293b-42f2-bfa7-5b0bc75a7881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:42.219598 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a2b3c-293b-42f2-bfa7-5b0bc75a7881 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.219885 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a2b3c-293b-42f2-bfa7-5b0bc75a7881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.220160 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a2b3c-293b-42f2-bfa7-5b0bc75a7881 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.220835 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a2b3c-293b-42f2-bfa7-5b0bc75a7881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:42.222847 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a2b3c-293b-42f2-bfa7-5b0bc75a7881 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:42.244560 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1342/2660] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:42 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:50:42.248980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a2b3c-293b-42f2-bfa7-5b0bc75a7881 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:42.262790 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a2b3c-293b-42f2-bfa7-5b0bc75a7881 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:42.263515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1a6ce-62c4-4e02-8f2b-a9f5a041ad4b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:42.267644 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f27eb6-c037-46bb-be31-e01b2e960493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:42.267874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320ad32-73ce-4130-a44a-e11829694a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:42.268841 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f27eb6-c037-46bb-be31-e01b2e960493 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.269054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320ad32-73ce-4130-a44a-e11829694a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.269246 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f27eb6-c037-46bb-be31-e01b2e960493 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.269347 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320ad32-73ce-4130-a44a-e11829694a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.269438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f27eb6-c037-46bb-be31-e01b2e960493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.269609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320ad32-73ce-4130-a44a-e11829694a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.270279 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320ad32-73ce-4130-a44a-e11829694a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:42.271884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320ad32-73ce-4130-a44a-e11829694a13 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:42.279815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f27eb6-c037-46bb-be31-e01b2e960493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:42.281326 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f27eb6-c037-46bb-be31-e01b2e960493 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:42.283190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1343/2661] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:42.288087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320ad32-73ce-4130-a44a-e11829694a13 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:42.292244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1319/2662] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:50:42.302290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320ad32-73ce-4130-a44a-e11829694a13 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:42.307251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78468b0-5a6a-4add-b84e-dd764487ac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:42.307939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78468b0-5a6a-4add-b84e-dd764487ac66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.308373 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78468b0-5a6a-4add-b84e-dd764487ac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.308502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78468b0-5a6a-4add-b84e-dd764487ac66 admin admin] PATH_INFO: `/v3/projects/20890163ebdf4460b35f094d70a91b8a/users/1fedfb33dafe4536b9bcba32846c406f/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.326068 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78468b0-5a6a-4add-b84e-dd764487ac66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20890163ebdf4460b35f094d70a91b8a, user_id=1fedfb33dafe4536b9bcba32846c406f, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:42.328816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78468b0-5a6a-4add-b84e-dd764487ac66 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:42.346706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1344/2663] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:42 2022] PUT /identity/v3/projects/20890163ebdf4460b35f094d70a91b8a/users/1fedfb33dafe4536b9bcba32846c406f/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:42.354179 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f27eb6-c037-46bb-be31-e01b2e960493 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:42.368466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f27eb6-c037-46bb-be31-e01b2e960493 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:42.444800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc45e3-a461-4e46-98fa-dea4ad6b2a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:42.445426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc45e3-a461-4e46-98fa-dea4ad6b2a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.445703 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc45e3-a461-4e46-98fa-dea4ad6b2a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.445977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc45e3-a461-4e46-98fa-dea4ad6b2a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.446706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc45e3-a461-4e46-98fa-dea4ad6b2a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:42.448201 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc45e3-a461-4e46-98fa-dea4ad6b2a26 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:42.461656 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1320/2664] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:42.465534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78468b0-5a6a-4add-b84e-dd764487ac66 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:42.479101 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78468b0-5a6a-4add-b84e-dd764487ac66 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:42.483963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff3494-c727-4329-8b1a-cacc7356c200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:42.484655 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff3494-c727-4329-8b1a-cacc7356c200 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.484937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff3494-c727-4329-8b1a-cacc7356c200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.485280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff3494-c727-4329-8b1a-cacc7356c200 admin admin] PATH_INFO: `/v3/projects/20890163ebdf4460b35f094d70a91b8a/users/1fedfb33dafe4536b9bcba32846c406f/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.488401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff3494-c727-4329-8b1a-cacc7356c200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20890163ebdf4460b35f094d70a91b8a, user_id=1fedfb33dafe4536b9bcba32846c406f, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:42.491098 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff3494-c727-4329-8b1a-cacc7356c200 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:42.506132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1345/2665] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:42 2022] PUT /identity/v3/projects/20890163ebdf4460b35f094d70a91b8a/users/1fedfb33dafe4536b9bcba32846c406f/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:42.514098 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94aa5e-1a65-4f4c-a39a-e7713109442d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.514388 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94aa5e-1a65-4f4c-a39a-e7713109442d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.514657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94aa5e-1a65-4f4c-a39a-e7713109442d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.548441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-5f94aa5e-1a65-4f4c-a39a-e7713109442d None None] MFA Rules not processed for user `1fedfb33dafe4536b9bcba32846c406f`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:42.550423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5f94aa5e-1a65-4f4c-a39a-e7713109442d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:42.584068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:42.613983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff3494-c727-4329-8b1a-cacc7356c200 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:42.615850 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5fff3494-c727-4329-8b1a-cacc7356c200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:42.641672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1321/2666] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:42.655857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94aa5e-1a65-4f4c-a39a-e7713109442d None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:42.657294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5f94aa5e-1a65-4f4c-a39a-e7713109442d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:42.700570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff3494-c727-4329-8b1a-cacc7356c200 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:42.706478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:42.707264 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.707612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.708215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] PATH_INFO: `/v3/projects/8894ac55d9fa46c1bbcc71e8140a8673` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.709308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8894ac55d9fa46c1bbcc71e8140a8673)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:42.711230 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:42.739189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94aa5e-1a65-4f4c-a39a-e7713109442d None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:42.743196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613bb67-2185-4b14-8408-136cf88d7665 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:42.743700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613bb67-2185-4b14-8408-136cf88d7665 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.743972 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613bb67-2185-4b14-8408-136cf88d7665 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.744265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613bb67-2185-4b14-8408-136cf88d7665 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.754897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613bb67-2185-4b14-8408-136cf88d7665 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:42.756854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613bb67-2185-4b14-8408-136cf88d7665 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:42.768058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1322/2667] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:42.779987 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8894ac55d9fa46c1bbcc71e8140a8673', 'request_id': 'req-351eaf13-0550-4b43-ad6c-7522c0950ea1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.786530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8894ac55d9fa46c1bbcc71e8140a8673', 'request_id': 'req-351eaf13-0550-4b43-ad6c-7522c0950ea1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.793940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8894ac55d9fa46c1bbcc71e8140a8673', 'request_id': 'req-351eaf13-0550-4b43-ad6c-7522c0950ea1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.796345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:42.812400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] The token cache is being invalidate because project 8894ac55d9fa46c1bbcc71e8140a8673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:42.812728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.813414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.815117 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1346/2668] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:42 2022] DELETE /identity/v3/projects/8894ac55d9fa46c1bbcc71e8140a8673 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:42.819148 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613bb67-2185-4b14-8408-136cf88d7665 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:42.823802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:42.825275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:42.830189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613bb67-2185-4b14-8408-136cf88d7665 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:42.891558 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351eaf13-0550-4b43-ad6c-7522c0950ea1 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:42.895405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39d68c-2f41-45ae-91f9-ccca2c5005cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:42.896013 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39d68c-2f41-45ae-91f9-ccca2c5005cc service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.896293 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39d68c-2f41-45ae-91f9-ccca2c5005cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.896501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:42.896635 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39d68c-2f41-45ae-91f9-ccca2c5005cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.897146 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:42.897433 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:42.897707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] PATH_INFO: `/v3/users/93e18abf16a84222b741e2dad70a16fb` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:42.898348 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8e39d68c-2f41-45ae-91f9-ccca2c5005cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:42.898847 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93e18abf16a84222b741e2dad70a16fb)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:42.900758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:42.967765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93e18abf16a84222b741e2dad70a16fb'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.977437 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93e18abf16a84222b741e2dad70a16fb'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.978172 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93e18abf16a84222b741e2dad70a16fb'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.978817 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93e18abf16a84222b741e2dad70a16fb'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.987771 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93e18abf16a84222b741e2dad70a16fb'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:42.990053 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39d68c-2f41-45ae-91f9-ccca2c5005cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:42.991584 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39d68c-2f41-45ae-91f9-ccca2c5005cc service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:42.996853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:43.002525 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1347/2669] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:50:43.021658 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1323/2670] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:42 2022] DELETE /identity/v3/users/93e18abf16a84222b741e2dad70a16fb => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:43.350852 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39d68c-2f41-45ae-91f9-ccca2c5005cc service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:43.352855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8e39d68c-2f41-45ae-91f9-ccca2c5005cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:43.379145 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39d68c-2f41-45ae-91f9-ccca2c5005cc service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:43.442035 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:43.442632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:43.442914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:43.443193 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] PATH_INFO: `/v3/projects/672edf23719e45aaa4407dc57fea2cc9` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:43.444376 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=672edf23719e45aaa4407dc57fea2cc9)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:43.446329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:43.494695 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:INSERT=1,keystone:DELETE=5,keystone:SELECT=94 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:43.496166 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '672edf23719e45aaa4407dc57fea2cc9', 'request_id': 'req-8bdbc396-cb35-40ef-94d3-018780ab83b6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:43.504962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '672edf23719e45aaa4407dc57fea2cc9', 'request_id': 'req-8bdbc396-cb35-40ef-94d3-018780ab83b6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:43.511961 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '672edf23719e45aaa4407dc57fea2cc9', 'request_id': 'req-8bdbc396-cb35-40ef-94d3-018780ab83b6'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:43.525353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] The token cache is being invalidate because project 672edf23719e45aaa4407dc57fea2cc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:43.525642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:43.526277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:43.527764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1348/2671] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:43 2022] DELETE /identity/v3/projects/672edf23719e45aaa4407dc57fea2cc9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:43.531834 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:43.533274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:43.546606 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65053b0a-3ada-4874-a137-a3ab59a3e1d8 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:43.606633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6baeb4-9bae-47a2-9db3-663e4d636cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:43.607266 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6baeb4-9bae-47a2-9db3-663e4d636cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:43.607543 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6baeb4-9bae-47a2-9db3-663e4d636cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:43.607815 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6baeb4-9bae-47a2-9db3-663e4d636cf5 admin admin] PATH_INFO: `/v3/users/fb8ad2f026454622832607c24c1d9da6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:43.608922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6baeb4-9bae-47a2-9db3-663e4d636cf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb8ad2f026454622832607c24c1d9da6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:43.610883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6baeb4-9bae-47a2-9db3-663e4d636cf5 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:43.656874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1b6baeb4-9bae-47a2-9db3-663e4d636cf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb8ad2f026454622832607c24c1d9da6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:43.663753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1b6baeb4-9bae-47a2-9db3-663e4d636cf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb8ad2f026454622832607c24c1d9da6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:43.664409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1b6baeb4-9bae-47a2-9db3-663e4d636cf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb8ad2f026454622832607c24c1d9da6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:43.665002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1b6baeb4-9bae-47a2-9db3-663e4d636cf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb8ad2f026454622832607c24c1d9da6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:43.672748 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-1b6baeb4-9bae-47a2-9db3-663e4d636cf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb8ad2f026454622832607c24c1d9da6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:43.694411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1324/2672] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:43 2022] DELETE /identity/v3/users/fb8ad2f026454622832607c24c1d9da6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:43.991616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:43.993047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:44.008878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbc396-cb35-40ef-94d3-018780ab83b6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:44.076192 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc06ea-9427-4828-9ded-800f0c02944f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:44.077324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc06ea-9427-4828-9ded-800f0c02944f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:44.077825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc06ea-9427-4828-9ded-800f0c02944f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:44.078335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc06ea-9427-4828-9ded-800f0c02944f admin admin] PATH_INFO: `/v3/projects/38ca8a6b996840b4835535742887ff8f` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:44.079869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc06ea-9427-4828-9ded-800f0c02944f admin admin] RBAC: Authorizing `identity:delete_project(project_id=38ca8a6b996840b4835535742887ff8f)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:44.081868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc06ea-9427-4828-9ded-800f0c02944f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:44.135722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b0fc06ea-9427-4828-9ded-800f0c02944f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38ca8a6b996840b4835535742887ff8f', 'request_id': 'req-b0fc06ea-9427-4828-9ded-800f0c02944f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:44.142144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b0fc06ea-9427-4828-9ded-800f0c02944f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38ca8a6b996840b4835535742887ff8f', 'request_id': 'req-b0fc06ea-9427-4828-9ded-800f0c02944f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:44.149952 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b0fc06ea-9427-4828-9ded-800f0c02944f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38ca8a6b996840b4835535742887ff8f', 'request_id': 'req-b0fc06ea-9427-4828-9ded-800f0c02944f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:44.163517 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b0fc06ea-9427-4828-9ded-800f0c02944f admin admin] The token cache is being invalidate because project 38ca8a6b996840b4835535742887ff8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:44.163811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b0fc06ea-9427-4828-9ded-800f0c02944f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:44.164434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b0fc06ea-9427-4828-9ded-800f0c02944f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:44.165985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1349/2673] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:43 2022] DELETE /identity/v3/projects/38ca8a6b996840b4835535742887ff8f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:44.172101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e430ef-0835-42bb-bfab-fdba54726ec8 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:44.172560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e430ef-0835-42bb-bfab-fdba54726ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:44.172671 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e430ef-0835-42bb-bfab-fdba54726ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:44.193662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-b9e430ef-0835-42bb-bfab-fdba54726ec8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:44.194886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b9e430ef-0835-42bb-bfab-fdba54726ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:44.258571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1325/2674] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:44.267162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1b4f9-9a3f-4370-89a9-698d60d54f55 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:44.267385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1b4f9-9a3f-4370-89a9-698d60d54f55 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:44.267662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1b4f9-9a3f-4370-89a9-698d60d54f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:44.288698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-a9b1b4f9-9a3f-4370-89a9-698d60d54f55 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:44.289953 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a9b1b4f9-9a3f-4370-89a9-698d60d54f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:44.298088 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1350/2675] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:44.302963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e430ef-0835-42bb-bfab-fdba54726ec8 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:44.312570 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e430ef-0835-42bb-bfab-fdba54726ec8 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:44.317492 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed67fb-ccde-4eaa-9c95-aa925802b270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:44.318201 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed67fb-ccde-4eaa-9c95-aa925802b270 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:44.318487 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed67fb-ccde-4eaa-9c95-aa925802b270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:44.318764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed67fb-ccde-4eaa-9c95-aa925802b270 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:44.319432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed67fb-ccde-4eaa-9c95-aa925802b270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:50:44.319737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed67fb-ccde-4eaa-9c95-aa925802b270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:44.321411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed67fb-ccde-4eaa-9c95-aa925802b270 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:44.324162 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:44.341215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1326/2676] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:50:44.345434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1b4f9-9a3f-4370-89a9-698d60d54f55 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:44.355117 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1b4f9-9a3f-4370-89a9-698d60d54f55 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:44.359947 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39735e13-caca-4b82-b1bc-b92a0b26eb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:44.360366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735e13-caca-4b82-b1bc-b92a0b26eb68 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:44.360644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735e13-caca-4b82-b1bc-b92a0b26eb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:44.360919 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39735e13-caca-4b82-b1bc-b92a0b26eb68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:44.362278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39735e13-caca-4b82-b1bc-b92a0b26eb68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:44.364248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39735e13-caca-4b82-b1bc-b92a0b26eb68 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:44.397073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1351/2677] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:44 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:44.401309 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed67fb-ccde-4eaa-9c95-aa925802b270 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:44.415604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed67fb-ccde-4eaa-9c95-aa925802b270 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:44.608306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfb3b8-1ccb-48f3-8caa-3d57e797a7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:44.608925 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfb3b8-1ccb-48f3-8caa-3d57e797a7af admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:44.609236 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfb3b8-1ccb-48f3-8caa-3d57e797a7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:44.609511 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfb3b8-1ccb-48f3-8caa-3d57e797a7af admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:44.610153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfb3b8-1ccb-48f3-8caa-3d57e797a7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:44.612112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfb3b8-1ccb-48f3-8caa-3d57e797a7af admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:44.636089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1327/2678] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:44 2022] POST /identity/v3/users => generated 295 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:44.640677 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39735e13-caca-4b82-b1bc-b92a0b26eb68 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:44.652903 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39735e13-caca-4b82-b1bc-b92a0b26eb68 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:44.657937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71364c9-5748-4880-b565-f4b3545bbd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:44.659155 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71364c9-5748-4880-b565-f4b3545bbd41 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:44.659413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71364c9-5748-4880-b565-f4b3545bbd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:44.659680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71364c9-5748-4880-b565-f4b3545bbd41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:44.660325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71364c9-5748-4880-b565-f4b3545bbd41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:44.661965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71364c9-5748-4880-b565-f4b3545bbd41 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:44.673204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1352/2679] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:44.677012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfb3b8-1ccb-48f3-8caa-3d57e797a7af admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:44.692613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfb3b8-1ccb-48f3-8caa-3d57e797a7af admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:44.698609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b98b2e-9645-40c7-9540-c53563376700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:44.699214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b98b2e-9645-40c7-9540-c53563376700 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:44.699490 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b98b2e-9645-40c7-9540-c53563376700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:44.699759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b98b2e-9645-40c7-9540-c53563376700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:44.700400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b98b2e-9645-40c7-9540-c53563376700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:44.702047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b98b2e-9645-40c7-9540-c53563376700 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:44.712915 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1328/2680] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:44.717233 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71364c9-5748-4880-b565-f4b3545bbd41 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:44.728844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71364c9-5748-4880-b565-f4b3545bbd41 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:44.733661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a3483-03d9-49e1-a34b-be8bdd33fd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:44.734477 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a3483-03d9-49e1-a34b-be8bdd33fd27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:44.734589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a3483-03d9-49e1-a34b-be8bdd33fd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:44.734830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a3483-03d9-49e1-a34b-be8bdd33fd27 admin admin] PATH_INFO: `/v3/projects/ae3d61655cad4b4b8458971974525f7d/users/a10e1bd018274db59ae3325d9e750686/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:44.753526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a3483-03d9-49e1-a34b-be8bdd33fd27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3d61655cad4b4b8458971974525f7d, user_id=a10e1bd018274db59ae3325d9e750686, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:44.757036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a3483-03d9-49e1-a34b-be8bdd33fd27 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:44.779360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1353/2681] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:44 2022] PUT /identity/v3/projects/ae3d61655cad4b4b8458971974525f7d/users/a10e1bd018274db59ae3325d9e750686/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:44.791974 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710fb00-6fa4-48b1-8a83-dae82c455181 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:44.792264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710fb00-6fa4-48b1-8a83-dae82c455181 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:44.792547 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710fb00-6fa4-48b1-8a83-dae82c455181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:44.835456 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-c710fb00-6fa4-48b1-8a83-dae82c455181 None None] MFA Rules not processed for user `a10e1bd018274db59ae3325d9e750686`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:44.836745 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c710fb00-6fa4-48b1-8a83-dae82c455181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:44.927219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1329/2682] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:44.931824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a3483-03d9-49e1-a34b-be8bdd33fd27 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:44.936307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:45.000292 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a3483-03d9-49e1-a34b-be8bdd33fd27 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:45.004956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e54bd1-235e-4dd2-85fc-7f435b9a444e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:45.005590 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e54bd1-235e-4dd2-85fc-7f435b9a444e admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:45.005866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e54bd1-235e-4dd2-85fc-7f435b9a444e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:45.006137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e54bd1-235e-4dd2-85fc-7f435b9a444e admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:45.006780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e54bd1-235e-4dd2-85fc-7f435b9a444e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:45.008797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e54bd1-235e-4dd2-85fc-7f435b9a444e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:45.036429 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1354/2683] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:44 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:45.042423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710fb00-6fa4-48b1-8a83-dae82c455181 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:45.059481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710fb00-6fa4-48b1-8a83-dae82c455181 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:45.116713 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca75658-1f64-4e17-ad1b-7a173098c591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:45.117349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca75658-1f64-4e17-ad1b-7a173098c591 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:45.117630 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca75658-1f64-4e17-ad1b-7a173098c591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:45.117904 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca75658-1f64-4e17-ad1b-7a173098c591 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:45.118525 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca75658-1f64-4e17-ad1b-7a173098c591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:45.120431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca75658-1f64-4e17-ad1b-7a173098c591 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:45.130364 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:45.142877 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1330/2684] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:45 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:45.146969 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e54bd1-235e-4dd2-85fc-7f435b9a444e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:45.157824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e54bd1-235e-4dd2-85fc-7f435b9a444e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:45.162999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febabd0c-a032-4d20-ab8c-c07ded2f1cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:45.163601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febabd0c-a032-4d20-ab8c-c07ded2f1cdf admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:45.163904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febabd0c-a032-4d20-ab8c-c07ded2f1cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:45.164180 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febabd0c-a032-4d20-ab8c-c07ded2f1cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:45.164823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febabd0c-a032-4d20-ab8c-c07ded2f1cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:45.166446 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febabd0c-a032-4d20-ab8c-c07ded2f1cdf admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:45.178081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1355/2685] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:45.182374 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca75658-1f64-4e17-ad1b-7a173098c591 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:45.192652 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca75658-1f64-4e17-ad1b-7a173098c591 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:45.197485 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a825d-1838-4db1-8d51-430a9085f4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:45.198127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a825d-1838-4db1-8d51-430a9085f4f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:45.198406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a825d-1838-4db1-8d51-430a9085f4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:45.198682 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a825d-1838-4db1-8d51-430a9085f4f4 admin admin] PATH_INFO: `/v3/projects/f9c80f04b6b24ec8ab3b01cae66a6f52/users/190f5dfd46964393a7b7a3dd8a00e68b/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:45.213795 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a825d-1838-4db1-8d51-430a9085f4f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c80f04b6b24ec8ab3b01cae66a6f52, user_id=190f5dfd46964393a7b7a3dd8a00e68b, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:45.216467 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a825d-1838-4db1-8d51-430a9085f4f4 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:45.230370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1331/2686] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:45 2022] PUT /identity/v3/projects/f9c80f04b6b24ec8ab3b01cae66a6f52/users/190f5dfd46964393a7b7a3dd8a00e68b/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:45.234405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febabd0c-a032-4d20-ab8c-c07ded2f1cdf admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:45.246616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febabd0c-a032-4d20-ab8c-c07ded2f1cdf admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:45.303305 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0ca59-8b74-4644-a679-46713e9ecb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:45.303839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0ca59-8b74-4644-a679-46713e9ecb23 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:45.304121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0ca59-8b74-4644-a679-46713e9ecb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:45.304396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0ca59-8b74-4644-a679-46713e9ecb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:45.305031 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0ca59-8b74-4644-a679-46713e9ecb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:45.306639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0ca59-8b74-4644-a679-46713e9ecb23 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:45.316353 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1356/2687] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:45.320526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a825d-1838-4db1-8d51-430a9085f4f4 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:45.331182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a825d-1838-4db1-8d51-430a9085f4f4 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:45.335688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79244b7b-5f1f-4607-b7db-900fa9b50f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:45.336324 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79244b7b-5f1f-4607-b7db-900fa9b50f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:45.336597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79244b7b-5f1f-4607-b7db-900fa9b50f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:45.336862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79244b7b-5f1f-4607-b7db-900fa9b50f63 admin admin] PATH_INFO: `/v3/projects/f9c80f04b6b24ec8ab3b01cae66a6f52/users/190f5dfd46964393a7b7a3dd8a00e68b/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:45.338773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79244b7b-5f1f-4607-b7db-900fa9b50f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c80f04b6b24ec8ab3b01cae66a6f52, user_id=190f5dfd46964393a7b7a3dd8a00e68b, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:45.341384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79244b7b-5f1f-4607-b7db-900fa9b50f63 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:45.352294 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1332/2688] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:45 2022] PUT /identity/v3/projects/f9c80f04b6b24ec8ab3b01cae66a6f52/users/190f5dfd46964393a7b7a3dd8a00e68b/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:45.362577 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7441b8-6d3b-48c3-a534-735c178b6161 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:45.362861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7441b8-6d3b-48c3-a534-735c178b6161 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:45.363162 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7441b8-6d3b-48c3-a534-735c178b6161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:45.389962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-fa7441b8-6d3b-48c3-a534-735c178b6161 None None] MFA Rules not processed for user `190f5dfd46964393a7b7a3dd8a00e68b`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:45.391154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-fa7441b8-6d3b-48c3-a534-735c178b6161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:45.458105 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:45.464693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1357/2689] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:45.475957 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79244b7b-5f1f-4607-b7db-900fa9b50f63 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:45.477717 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-79244b7b-5f1f-4607-b7db-900fa9b50f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:45.535477 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79244b7b-5f1f-4607-b7db-900fa9b50f63 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:45.539160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df7801-7af3-4171-9da3-2676937de8b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:45.539763 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df7801-7af3-4171-9da3-2676937de8b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:45.540044 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df7801-7af3-4171-9da3-2676937de8b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:45.540319 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df7801-7af3-4171-9da3-2676937de8b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:45.549572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df7801-7af3-4171-9da3-2676937de8b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:45.551024 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df7801-7af3-4171-9da3-2676937de8b8 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:45.560818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1333/2690] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:45.699939 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7441b8-6d3b-48c3-a534-735c178b6161 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:45.762096 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7441b8-6d3b-48c3-a534-735c178b6161 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:45.767040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:45.768222 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:45.768498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:45.768768 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] PATH_INFO: `/v3/users/a10e1bd018274db59ae3325d9e750686` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:45.769924 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] RBAC: Authorizing `identity:delete_user(user_id=a10e1bd018274db59ae3325d9e750686)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:45.771824 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:45.823266 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a10e1bd018274db59ae3325d9e750686'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:45.835567 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a10e1bd018274db59ae3325d9e750686'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:45.836241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a10e1bd018274db59ae3325d9e750686'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:45.843091 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a10e1bd018274db59ae3325d9e750686'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:45.843733 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a10e1bd018274db59ae3325d9e750686'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:45.856841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1358/2691] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:45 2022] DELETE /identity/v3/users/a10e1bd018274db59ae3325d9e750686 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:45.861599 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f5cdf-ea6e-4306-949e-85b44e7ad416 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:45.861898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f5cdf-ea6e-4306-949e-85b44e7ad416 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:45.862197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f5cdf-ea6e-4306-949e-85b44e7ad416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:45.882451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-427f5cdf-ea6e-4306-949e-85b44e7ad416 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:45.883562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-427f5cdf-ea6e-4306-949e-85b44e7ad416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:45.947357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1334/2692] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:45.957819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:45.959242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:46.021476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3e120-edff-4f15-9f5d-b5fad0de11da admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:46.025290 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c287df7-853b-44cd-8a34-54ee9be57ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:46.025903 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c287df7-853b-44cd-8a34-54ee9be57ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:46.026186 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c287df7-853b-44cd-8a34-54ee9be57ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:46.026486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c287df7-853b-44cd-8a34-54ee9be57ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:46.038697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c287df7-853b-44cd-8a34-54ee9be57ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:46.040112 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c287df7-853b-44cd-8a34-54ee9be57ed2 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:46.051197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1359/2693] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:50:46.356718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f5cdf-ea6e-4306-949e-85b44e7ad416 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:46.358263 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-427f5cdf-ea6e-4306-949e-85b44e7ad416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:46.373344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f5cdf-ea6e-4306-949e-85b44e7ad416 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:46.379391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:46.380684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:46.381229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:46.381587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] PATH_INFO: `/v3/projects/ae3d61655cad4b4b8458971974525f7d` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:46.383142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3d61655cad4b4b8458971974525f7d)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:46.386461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:46.440488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3d61655cad4b4b8458971974525f7d', 'request_id': 'req-3363cfe1-7a4d-4f16-859c-ee489225d36c'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.444664 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:46.447167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3d61655cad4b4b8458971974525f7d', 'request_id': 'req-3363cfe1-7a4d-4f16-859c-ee489225d36c'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.455678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3d61655cad4b4b8458971974525f7d', 'request_id': 'req-3363cfe1-7a4d-4f16-859c-ee489225d36c'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.467175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] The token cache is being invalidate because project ae3d61655cad4b4b8458971974525f7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:46.467526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.468165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.469778 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1335/2694] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:46 2022] DELETE /identity/v3/projects/ae3d61655cad4b4b8458971974525f7d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:46.474314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c287df7-853b-44cd-8a34-54ee9be57ed2 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:46.484587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c287df7-853b-44cd-8a34-54ee9be57ed2 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:46.503312 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:46.521308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:46.522905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:46.556668 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114acb5-0c30-4a4b-97f1-971804bcb992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:46.557354 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114acb5-0c30-4a4b-97f1-971804bcb992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:46.557762 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114acb5-0c30-4a4b-97f1-971804bcb992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:46.558080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114acb5-0c30-4a4b-97f1-971804bcb992 admin admin] PATH_INFO: `/v3/users/190f5dfd46964393a7b7a3dd8a00e68b` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:46.559264 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114acb5-0c30-4a4b-97f1-971804bcb992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=190f5dfd46964393a7b7a3dd8a00e68b)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:46.561214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114acb5-0c30-4a4b-97f1-971804bcb992 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:46.591992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363cfe1-7a4d-4f16-859c-ee489225d36c admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:46.597227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:46.597859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:46.598129 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:46.598393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] PATH_INFO: `/v3/users/ab4d7e2204074c78bf118eba7c9b3d6a` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:46.599526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab4d7e2204074c78bf118eba7c9b3d6a)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:46.601462 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:46.616999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8114acb5-0c30-4a4b-97f1-971804bcb992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '190f5dfd46964393a7b7a3dd8a00e68b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.629774 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8114acb5-0c30-4a4b-97f1-971804bcb992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '190f5dfd46964393a7b7a3dd8a00e68b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.630505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8114acb5-0c30-4a4b-97f1-971804bcb992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '190f5dfd46964393a7b7a3dd8a00e68b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.638335 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8114acb5-0c30-4a4b-97f1-971804bcb992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '190f5dfd46964393a7b7a3dd8a00e68b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.639000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-8114acb5-0c30-4a4b-97f1-971804bcb992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '190f5dfd46964393a7b7a3dd8a00e68b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.650527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab4d7e2204074c78bf118eba7c9b3d6a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.653771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1360/2695] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:46 2022] DELETE /identity/v3/users/190f5dfd46964393a7b7a3dd8a00e68b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:46.659485 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab4d7e2204074c78bf118eba7c9b3d6a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.660191 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab4d7e2204074c78bf118eba7c9b3d6a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.660878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab4d7e2204074c78bf118eba7c9b3d6a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.670167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab4d7e2204074c78bf118eba7c9b3d6a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:46.691415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1336/2696] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:46 2022] DELETE /identity/v3/users/ab4d7e2204074c78bf118eba7c9b3d6a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:46.696322 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb909c-385f-4abc-b8ea-b375660468da None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:46.696614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb909c-385f-4abc-b8ea-b375660468da None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:46.696896 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb909c-385f-4abc-b8ea-b375660468da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:46.718636 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-22bb909c-385f-4abc-b8ea-b375660468da None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:46.720014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-22bb909c-385f-4abc-b8ea-b375660468da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:46.787452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1361/2697] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:46.797447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:46.799512 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:46.871924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5082e-3e2d-4f00-bf6a-4bcf4bd81c89 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:46.875674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e7b60-b821-482b-bfb5-fa7801b93257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:46.876342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7b60-b821-482b-bfb5-fa7801b93257 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:46.876547 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7b60-b821-482b-bfb5-fa7801b93257 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:46.876820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7b60-b821-482b-bfb5-fa7801b93257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:46.886346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e7b60-b821-482b-bfb5-fa7801b93257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:46.887892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e7b60-b821-482b-bfb5-fa7801b93257 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:46.900021 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1337/2698] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:46.973592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb909c-385f-4abc-b8ea-b375660468da None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:46.978217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-22bb909c-385f-4abc-b8ea-b375660468da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:46.993055 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb909c-385f-4abc-b8ea-b375660468da None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.000258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.001189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.001967 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.002093 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b admin admin] PATH_INFO: `/v3/projects/f9c80f04b6b24ec8ab3b01cae66a6f52` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.003576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c80f04b6b24ec8ab3b01cae66a6f52)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.005866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.050692 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:47.063775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c80f04b6b24ec8ab3b01cae66a6f52', 'request_id': 'req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.069878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c80f04b6b24ec8ab3b01cae66a6f52', 'request_id': 'req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.076633 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c80f04b6b24ec8ab3b01cae66a6f52', 'request_id': 'req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.095144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b admin admin] The token cache is being invalidate because project f9c80f04b6b24ec8ab3b01cae66a6f52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:47.095465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.096705 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-cdf73551-3360-43c8-b1e8-5c8fa63ea42b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.098518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1362/2699] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:46 2022] DELETE /identity/v3/projects/f9c80f04b6b24ec8ab3b01cae66a6f52 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:47.103064 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a454e9-4f9c-49ba-a696-de1f26a44d43 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.103352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a454e9-4f9c-49ba-a696-de1f26a44d43 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.103621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a454e9-4f9c-49ba-a696-de1f26a44d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.112872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:47.125580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-01a454e9-4f9c-49ba-a696-de1f26a44d43 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:47.127517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-01a454e9-4f9c-49ba-a696-de1f26a44d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:47.191276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1338/2700] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:47.199101 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1d06-fba1-4a9d-921e-b0c2a2d600f4 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.199384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1d06-fba1-4a9d-921e-b0c2a2d600f4 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.199654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef1d06-fba1-4a9d-921e-b0c2a2d600f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.221481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-daef1d06-fba1-4a9d-921e-b0c2a2d600f4 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:47.222816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-daef1d06-fba1-4a9d-921e-b0c2a2d600f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:47.231244 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1363/2701] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:47.235068 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef1d06-fba1-4a9d-921e-b0c2a2d600f4 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.235787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a454e9-4f9c-49ba-a696-de1f26a44d43 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.236582 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-daef1d06-fba1-4a9d-921e-b0c2a2d600f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:47.248036 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a454e9-4f9c-49ba-a696-de1f26a44d43 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.250881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef1d06-fba1-4a9d-921e-b0c2a2d600f4 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.254582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa76ad-075f-4afa-a8f8-c8f7eec51ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.255226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa76ad-075f-4afa-a8f8-c8f7eec51ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.255514 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa76ad-075f-4afa-a8f8-c8f7eec51ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.255787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa76ad-075f-4afa-a8f8-c8f7eec51ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.256502 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa76ad-075f-4afa-a8f8-c8f7eec51ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:50:47.256790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa76ad-075f-4afa-a8f8-c8f7eec51ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.257749 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.258389 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.258526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa76ad-075f-4afa-a8f8-c8f7eec51ccf admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.258688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.258940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] PATH_INFO: `/v3/projects/e13b964ffdf045f7be92eadd25e35a28` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.260319 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e13b964ffdf045f7be92eadd25e35a28)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.262803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.272478 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1339/2702] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:50:47.276955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa76ad-075f-4afa-a8f8-c8f7eec51ccf admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.286946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa76ad-075f-4afa-a8f8-c8f7eec51ccf admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.291617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f8e03-d080-494d-8c39-caf855572fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.292471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f8e03-d080-494d-8c39-caf855572fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.292916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f8e03-d080-494d-8c39-caf855572fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.293227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f8e03-d080-494d-8c39-caf855572fb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.294140 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f8e03-d080-494d-8c39-caf855572fb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.296584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f8e03-d080-494d-8c39-caf855572fb5 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.315029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e13b964ffdf045f7be92eadd25e35a28', 'request_id': 'req-18481c08-da3e-409c-aeba-cb3bd66363a3'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.321552 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e13b964ffdf045f7be92eadd25e35a28', 'request_id': 'req-18481c08-da3e-409c-aeba-cb3bd66363a3'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.325171 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1340/2703] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:47 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:47.328281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e13b964ffdf045f7be92eadd25e35a28', 'request_id': 'req-18481c08-da3e-409c-aeba-cb3bd66363a3'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.329042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f8e03-d080-494d-8c39-caf855572fb5 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.343138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] The token cache is being invalidate because project e13b964ffdf045f7be92eadd25e35a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:47.343437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.344082 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.346452 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1364/2704] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:47 2022] DELETE /identity/v3/projects/e13b964ffdf045f7be92eadd25e35a28 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:47.350037 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.351459 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:47.393704 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f8e03-d080-494d-8c39-caf855572fb5 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.398462 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da0b0f-c21c-4ea9-b163-066760e0d84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.399066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da0b0f-c21c-4ea9-b163-066760e0d84c admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.399345 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da0b0f-c21c-4ea9-b163-066760e0d84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.399623 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da0b0f-c21c-4ea9-b163-066760e0d84c admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.400251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da0b0f-c21c-4ea9-b163-066760e0d84c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.402278 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da0b0f-c21c-4ea9-b163-066760e0d84c admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.415579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18481c08-da3e-409c-aeba-cb3bd66363a3 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.421874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece381c4-215b-4db5-a840-866cdc86aef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.422502 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece381c4-215b-4db5-a840-866cdc86aef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.422783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece381c4-215b-4db5-a840-866cdc86aef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.423057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece381c4-215b-4db5-a840-866cdc86aef1 admin admin] PATH_INFO: `/v3/users/1fedfb33dafe4536b9bcba32846c406f` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.424728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece381c4-215b-4db5-a840-866cdc86aef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fedfb33dafe4536b9bcba32846c406f)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.425926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1341/2705] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:47 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:50:47.426666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece381c4-215b-4db5-a840-866cdc86aef1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.429987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da0b0f-c21c-4ea9-b163-066760e0d84c admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.431639 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-00da0b0f-c21c-4ea9-b163-066760e0d84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:47.443796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da0b0f-c21c-4ea9-b163-066760e0d84c admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.448776 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161b493-3263-40e7-8480-2e2bc7c789ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.449411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161b493-3263-40e7-8480-2e2bc7c789ea admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.449697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161b493-3263-40e7-8480-2e2bc7c789ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.449970 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161b493-3263-40e7-8480-2e2bc7c789ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.450614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161b493-3263-40e7-8480-2e2bc7c789ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.452295 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161b493-3263-40e7-8480-2e2bc7c789ea admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.463106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1342/2706] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:47.467138 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161b493-3263-40e7-8480-2e2bc7c789ea admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.476187 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:47.477588 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161b493-3263-40e7-8480-2e2bc7c789ea admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.482561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ece381c4-215b-4db5-a840-866cdc86aef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fedfb33dafe4536b9bcba32846c406f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.483320 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eaa0ad-69dd-42f4-8aeb-6426f37962b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.483924 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eaa0ad-69dd-42f4-8aeb-6426f37962b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.484211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eaa0ad-69dd-42f4-8aeb-6426f37962b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.484489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eaa0ad-69dd-42f4-8aeb-6426f37962b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.485878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eaa0ad-69dd-42f4-8aeb-6426f37962b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.488077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eaa0ad-69dd-42f4-8aeb-6426f37962b5 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.496836 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ece381c4-215b-4db5-a840-866cdc86aef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fedfb33dafe4536b9bcba32846c406f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.497586 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ece381c4-215b-4db5-a840-866cdc86aef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fedfb33dafe4536b9bcba32846c406f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.499737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1343/2707] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:47.503836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eaa0ad-69dd-42f4-8aeb-6426f37962b5 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.505205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ece381c4-215b-4db5-a840-866cdc86aef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fedfb33dafe4536b9bcba32846c406f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.505666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-96eaa0ad-69dd-42f4-8aeb-6426f37962b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:47.505871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ece381c4-215b-4db5-a840-866cdc86aef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fedfb33dafe4536b9bcba32846c406f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.523927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eaa0ad-69dd-42f4-8aeb-6426f37962b5 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.526600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1365/2708] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:47 2022] DELETE /identity/v3/users/1fedfb33dafe4536b9bcba32846c406f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:47.531122 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493c8ee-a7f2-4578-8425-f7ba3de6857a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.531794 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493c8ee-a7f2-4578-8425-f7ba3de6857a admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.532076 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493c8ee-a7f2-4578-8425-f7ba3de6857a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.532352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493c8ee-a7f2-4578-8425-f7ba3de6857a admin admin] PATH_INFO: `/v3/projects/17f582c9edbc4187a62e356226e53872/users/c3441131a52b40a0bca0dee5e7bb7c7f/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.548464 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493c8ee-a7f2-4578-8425-f7ba3de6857a admin admin] RBAC: Authorizing `identity:create_grant(project_id=17f582c9edbc4187a62e356226e53872, user_id=c3441131a52b40a0bca0dee5e7bb7c7f, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.552719 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493c8ee-a7f2-4578-8425-f7ba3de6857a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.567310 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1344/2709] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:47 2022] PUT /identity/v3/projects/17f582c9edbc4187a62e356226e53872/users/c3441131a52b40a0bca0dee5e7bb7c7f/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:47.578951 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17d84-9b03-403d-ad17-18357c4147f3 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.579605 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17d84-9b03-403d-ad17-18357c4147f3 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.580449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17d84-9b03-403d-ad17-18357c4147f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.614019 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-a2c17d84-9b03-403d-ad17-18357c4147f3 None None] MFA Rules not processed for user `c3441131a52b40a0bca0dee5e7bb7c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:47.615280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a2c17d84-9b03-403d-ad17-18357c4147f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:47.683469 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1366/2710] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:47.688074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493c8ee-a7f2-4578-8425-f7ba3de6857a admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.701960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:47.755971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493c8ee-a7f2-4578-8425-f7ba3de6857a admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.760474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd123c0d-8799-4ee0-8493-267a513ab821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.761123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd123c0d-8799-4ee0-8493-267a513ab821 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.761392 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd123c0d-8799-4ee0-8493-267a513ab821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.761666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd123c0d-8799-4ee0-8493-267a513ab821 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.762315 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd123c0d-8799-4ee0-8493-267a513ab821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.764244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd123c0d-8799-4ee0-8493-267a513ab821 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.794042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1345/2711] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:47 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:47.800020 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c17d84-9b03-403d-ad17-18357c4147f3 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.811913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c17d84-9b03-403d-ad17-18357c4147f3 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.819604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd123c0d-8799-4ee0-8493-267a513ab821 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.821792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-bd123c0d-8799-4ee0-8493-267a513ab821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:47.879431 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd69d55-a26c-48f1-a979-41441214e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.880174 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd69d55-a26c-48f1-a979-41441214e9ad admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.880455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd69d55-a26c-48f1-a979-41441214e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.880728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd69d55-a26c-48f1-a979-41441214e9ad admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.881406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd69d55-a26c-48f1-a979-41441214e9ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.883441 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd69d55-a26c-48f1-a979-41441214e9ad admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.904759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd123c0d-8799-4ee0-8493-267a513ab821 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.907965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1367/2712] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:47 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:47.910213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5dcef-7822-4cfc-a037-3ac8c09280e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.910846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5dcef-7822-4cfc-a037-3ac8c09280e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.911489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5dcef-7822-4cfc-a037-3ac8c09280e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.911489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5dcef-7822-4cfc-a037-3ac8c09280e1 admin admin] PATH_INFO: `/v3/projects/20890163ebdf4460b35f094d70a91b8a` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.912321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd69d55-a26c-48f1-a979-41441214e9ad admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.912697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5dcef-7822-4cfc-a037-3ac8c09280e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20890163ebdf4460b35f094d70a91b8a)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.914741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5dcef-7822-4cfc-a037-3ac8c09280e1 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.922513 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd69d55-a26c-48f1-a979-41441214e9ad admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.927122 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372babf-74a2-41e0-8500-9af80e2cdda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:47.927712 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372babf-74a2-41e0-8500-9af80e2cdda0 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:47.927987 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372babf-74a2-41e0-8500-9af80e2cdda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:47.928263 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372babf-74a2-41e0-8500-9af80e2cdda0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:47.928899 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372babf-74a2-41e0-8500-9af80e2cdda0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:47.930510 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372babf-74a2-41e0-8500-9af80e2cdda0 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:47.940904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1368/2713] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:47.945539 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372babf-74a2-41e0-8500-9af80e2cdda0 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:47.956651 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372babf-74a2-41e0-8500-9af80e2cdda0 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:47.962408 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-86d5dcef-7822-4cfc-a037-3ac8c09280e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20890163ebdf4460b35f094d70a91b8a', 'request_id': 'req-86d5dcef-7822-4cfc-a037-3ac8c09280e1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.970029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-86d5dcef-7822-4cfc-a037-3ac8c09280e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20890163ebdf4460b35f094d70a91b8a', 'request_id': 'req-86d5dcef-7822-4cfc-a037-3ac8c09280e1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.979830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-86d5dcef-7822-4cfc-a037-3ac8c09280e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20890163ebdf4460b35f094d70a91b8a', 'request_id': 'req-86d5dcef-7822-4cfc-a037-3ac8c09280e1'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.982574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:47.995126 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-86d5dcef-7822-4cfc-a037-3ac8c09280e1 admin admin] The token cache is being invalidate because project 20890163ebdf4460b35f094d70a91b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:47.995365 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-86d5dcef-7822-4cfc-a037-3ac8c09280e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.996065 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-86d5dcef-7822-4cfc-a037-3ac8c09280e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:47.997840 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1346/2714] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:47 2022] DELETE /identity/v3/projects/20890163ebdf4460b35f094d70a91b8a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:48.003651 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abe25a-d578-4e4b-a30c-8d3f8fc33046 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.003933 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abe25a-d578-4e4b-a30c-8d3f8fc33046 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.004203 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abe25a-d578-4e4b-a30c-8d3f8fc33046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.023797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-66abe25a-d578-4e4b-a30c-8d3f8fc33046 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:48.025010 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-66abe25a-d578-4e4b-a30c-8d3f8fc33046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:48.025787 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4b24a-92fa-4b89-96c5-f80519e7f385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.026480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4b24a-92fa-4b89-96c5-f80519e7f385 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.026808 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4b24a-92fa-4b89-96c5-f80519e7f385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.027203 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4b24a-92fa-4b89-96c5-f80519e7f385 admin admin] PATH_INFO: `/v3/projects/772220a6b6744b05902864caa34d4a20/users/4fba4503c03c460881c008fdb50a3ea6/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.033990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1347/2715] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:48.042580 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40ad4b-a71c-4f8c-87fc-9b76582417d5 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.042869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40ad4b-a71c-4f8c-87fc-9b76582417d5 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.043134 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40ad4b-a71c-4f8c-87fc-9b76582417d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.052132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4b24a-92fa-4b89-96c5-f80519e7f385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=772220a6b6744b05902864caa34d4a20, user_id=4fba4503c03c460881c008fdb50a3ea6, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.054807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4b24a-92fa-4b89-96c5-f80519e7f385 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.063399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ff40ad4b-a71c-4f8c-87fc-9b76582417d5 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:48.065620 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ff40ad4b-a71c-4f8c-87fc-9b76582417d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:48.069635 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1369/2716] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:47 2022] PUT /identity/v3/projects/772220a6b6744b05902864caa34d4a20/users/4fba4503c03c460881c008fdb50a3ea6/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:48.073631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4b24a-92fa-4b89-96c5-f80519e7f385 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.075412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-23f4b24a-92fa-4b89-96c5-f80519e7f385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:48.084007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:48.128736 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1348/2717] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:48.133321 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40ad4b-a71c-4f8c-87fc-9b76582417d5 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.143601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40ad4b-a71c-4f8c-87fc-9b76582417d5 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.144728 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4b24a-92fa-4b89-96c5-f80519e7f385 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.148554 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380fc6-ebad-4f5b-ab76-dd33087f8681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.149302 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380fc6-ebad-4f5b-ab76-dd33087f8681 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.149423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa354f-cfe7-4ee4-ad51-3d577733b090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.149611 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380fc6-ebad-4f5b-ab76-dd33087f8681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.149861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380fc6-ebad-4f5b-ab76-dd33087f8681 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.150002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa354f-cfe7-4ee4-ad51-3d577733b090 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.150256 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa354f-cfe7-4ee4-ad51-3d577733b090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.150505 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf380fc6-ebad-4f5b-ab76-dd33087f8681 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:50:48.150625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa354f-cfe7-4ee4-ad51-3d577733b090 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.150808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf380fc6-ebad-4f5b-ab76-dd33087f8681 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.151153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa354f-cfe7-4ee4-ad51-3d577733b090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.152396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf380fc6-ebad-4f5b-ab76-dd33087f8681 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.152718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fa354f-cfe7-4ee4-ad51-3d577733b090 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.166710 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1370/2718] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:48.166896 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1349/2719] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:50:48.169323 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380fc6-ebad-4f5b-ab76-dd33087f8681 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.169448 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa354f-cfe7-4ee4-ad51-3d577733b090 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.180481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa354f-cfe7-4ee4-ad51-3d577733b090 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.181151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380fc6-ebad-4f5b-ab76-dd33087f8681 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.185343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49d9c6-0d02-415c-85a4-ded946500d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.185683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61eb1b8-483e-45c7-ae02-1f1c1c981a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.186210 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49d9c6-0d02-415c-85a4-ded946500d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.186334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eb1b8-483e-45c7-ae02-1f1c1c981a95 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.186549 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eb1b8-483e-45c7-ae02-1f1c1c981a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.186848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49d9c6-0d02-415c-85a4-ded946500d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.186958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eb1b8-483e-45c7-ae02-1f1c1c981a95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.187343 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49d9c6-0d02-415c-85a4-ded946500d9d admin admin] PATH_INFO: `/v3/projects/772220a6b6744b05902864caa34d4a20/users/4fba4503c03c460881c008fdb50a3ea6/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.187481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61eb1b8-483e-45c7-ae02-1f1c1c981a95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.189430 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61eb1b8-483e-45c7-ae02-1f1c1c981a95 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.189664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49d9c6-0d02-415c-85a4-ded946500d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=772220a6b6744b05902864caa34d4a20, user_id=4fba4503c03c460881c008fdb50a3ea6, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.192371 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49d9c6-0d02-415c-85a4-ded946500d9d admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.203294 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1371/2720] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:48 2022] PUT /identity/v3/projects/772220a6b6744b05902864caa34d4a20/users/4fba4503c03c460881c008fdb50a3ea6/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:48.213629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d03fcc-210e-41bc-aaf4-553013233cff None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.213916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d03fcc-210e-41bc-aaf4-553013233cff None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.214190 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d03fcc-210e-41bc-aaf4-553013233cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.221230 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1350/2721] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:48 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:48.222154 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61eb1b8-483e-45c7-ae02-1f1c1c981a95 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.243522 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-11d03fcc-210e-41bc-aaf4-553013233cff None None] MFA Rules not processed for user `4fba4503c03c460881c008fdb50a3ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:48.244774 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-11d03fcc-210e-41bc-aaf4-553013233cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:48.286080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61eb1b8-483e-45c7-ae02-1f1c1c981a95 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.290767 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b847d3-a550-4aae-928a-65f9cdc5613b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.291414 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b847d3-a550-4aae-928a-65f9cdc5613b admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.291622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b847d3-a550-4aae-928a-65f9cdc5613b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.291891 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b847d3-a550-4aae-928a-65f9cdc5613b admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.292513 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b847d3-a550-4aae-928a-65f9cdc5613b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.294978 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b847d3-a550-4aae-928a-65f9cdc5613b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.314926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1372/2722] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:48.323518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1351/2723] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:48 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:48.324463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d03fcc-210e-41bc-aaf4-553013233cff None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.325591 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b847d3-a550-4aae-928a-65f9cdc5613b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.327276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d6b847d3-a550-4aae-928a-65f9cdc5613b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:48.335394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d03fcc-210e-41bc-aaf4-553013233cff None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.340471 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4b9a0-87b9-45fc-ab7f-4a170dee033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.341066 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4b9a0-87b9-45fc-ab7f-4a170dee033e admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.341372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4b9a0-87b9-45fc-ab7f-4a170dee033e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.341766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4b9a0-87b9-45fc-ab7f-4a170dee033e admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.342400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4b9a0-87b9-45fc-ab7f-4a170dee033e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.343972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4b9a0-87b9-45fc-ab7f-4a170dee033e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.354881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1373/2724] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:48.358737 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4b9a0-87b9-45fc-ab7f-4a170dee033e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.369988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4b9a0-87b9-45fc-ab7f-4a170dee033e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.374936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e9d15-9432-4b2c-b756-7a9eec31497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.375727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e9d15-9432-4b2c-b756-7a9eec31497e admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.376004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e9d15-9432-4b2c-b756-7a9eec31497e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.376272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e9d15-9432-4b2c-b756-7a9eec31497e admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.377000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e9d15-9432-4b2c-b756-7a9eec31497e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.378604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e9d15-9432-4b2c-b756-7a9eec31497e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.379926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:48.385110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b847d3-a550-4aae-928a-65f9cdc5613b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.389429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9346b3-0607-4148-91d7-d71bc860ad75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.390108 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9346b3-0607-4148-91d7-d71bc860ad75 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.390457 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9346b3-0607-4148-91d7-d71bc860ad75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.390870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9346b3-0607-4148-91d7-d71bc860ad75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.393200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1374/2725] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:48.394719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e9d15-9432-4b2c-b756-7a9eec31497e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.401660 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9346b3-0607-4148-91d7-d71bc860ad75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.403142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9346b3-0607-4148-91d7-d71bc860ad75 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.404616 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e9d15-9432-4b2c-b756-7a9eec31497e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.409911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a966cd-614f-415f-badb-27cd7c090c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.410556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a966cd-614f-415f-badb-27cd7c090c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.410834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a966cd-614f-415f-badb-27cd7c090c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.411107 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a966cd-614f-415f-badb-27cd7c090c8b admin admin] PATH_INFO: `/v3/projects/a6423c6ffc56441a88fe59581fad6e9d/users/044bb907ca2748f7a171dc0e1da1e0db/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.413490 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1352/2726] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:48.427755 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a966cd-614f-415f-badb-27cd7c090c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6423c6ffc56441a88fe59581fad6e9d, user_id=044bb907ca2748f7a171dc0e1da1e0db, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.430422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a966cd-614f-415f-badb-27cd7c090c8b admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.450391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1375/2727] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:48 2022] PUT /identity/v3/projects/a6423c6ffc56441a88fe59581fad6e9d/users/044bb907ca2748f7a171dc0e1da1e0db/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:48.454557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc40675-edb1-4385-8f54-6c36a2a33bd0 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.455475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc40675-edb1-4385-8f54-6c36a2a33bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.455757 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc40675-edb1-4385-8f54-6c36a2a33bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.487407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-3fc40675-edb1-4385-8f54-6c36a2a33bd0 None None] MFA Rules not processed for user `044bb907ca2748f7a171dc0e1da1e0db`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:48.488617 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3fc40675-edb1-4385-8f54-6c36a2a33bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:48.498402 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:48.565674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1353/2728] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:48 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:48.570714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a966cd-614f-415f-badb-27cd7c090c8b admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.643689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a966cd-614f-415f-badb-27cd7c090c8b admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.648577 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe8eb8-336f-46fc-9d97-ef798b6d2cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.649217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe8eb8-336f-46fc-9d97-ef798b6d2cfb admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.649501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe8eb8-336f-46fc-9d97-ef798b6d2cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.649772 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe8eb8-336f-46fc-9d97-ef798b6d2cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.650412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe8eb8-336f-46fc-9d97-ef798b6d2cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.652351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe8eb8-336f-46fc-9d97-ef798b6d2cfb admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.682216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1376/2729] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:48 2022] POST /identity/v3/projects => generated 397 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:50:48.686513 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc40675-edb1-4385-8f54-6c36a2a33bd0 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.698393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc40675-edb1-4385-8f54-6c36a2a33bd0 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.769422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49091f01-4e8d-4146-bbdc-8f56415f7967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.770060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49091f01-4e8d-4146-bbdc-8f56415f7967 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.770338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49091f01-4e8d-4146-bbdc-8f56415f7967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.770608 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49091f01-4e8d-4146-bbdc-8f56415f7967 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.771270 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49091f01-4e8d-4146-bbdc-8f56415f7967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.773317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49091f01-4e8d-4146-bbdc-8f56415f7967 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.802069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1354/2730] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:48 2022] POST /identity/v3/users => generated 305 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:50:48.804364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe8eb8-336f-46fc-9d97-ef798b6d2cfb admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.812352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49091f01-4e8d-4146-bbdc-8f56415f7967 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.823427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe8eb8-336f-46fc-9d97-ef798b6d2cfb admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.823931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49091f01-4e8d-4146-bbdc-8f56415f7967 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.828342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2deb7a-238c-463a-af77-df47dee0961a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.828789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc59cfd-4846-49b1-bc83-ccddace91ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.828914 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2deb7a-238c-463a-af77-df47dee0961a admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.829165 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2deb7a-238c-463a-af77-df47dee0961a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.829445 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2deb7a-238c-463a-af77-df47dee0961a admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.829571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc59cfd-4846-49b1-bc83-ccddace91ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.829722 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc59cfd-4846-49b1-bc83-ccddace91ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.830002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc59cfd-4846-49b1-bc83-ccddace91ae3 admin admin] PATH_INFO: `/v3/users/c3441131a52b40a0bca0dee5e7bb7c7f` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.830121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2deb7a-238c-463a-af77-df47dee0961a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.831426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc59cfd-4846-49b1-bc83-ccddace91ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3441131a52b40a0bca0dee5e7bb7c7f)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.831792 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2deb7a-238c-463a-af77-df47dee0961a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.833359 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc59cfd-4846-49b1-bc83-ccddace91ae3 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.843449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1377/2731] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:48.848730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2deb7a-238c-463a-af77-df47dee0961a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.858765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2deb7a-238c-463a-af77-df47dee0961a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:48.865252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a83dc6-1713-49b8-9701-fc44a5dd6d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:48.865962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a83dc6-1713-49b8-9701-fc44a5dd6d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.866304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a83dc6-1713-49b8-9701-fc44a5dd6d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.866625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a83dc6-1713-49b8-9701-fc44a5dd6d1c admin admin] PATH_INFO: `/v3/projects/34e3068962a343d79494c5a846cf4b6a/users/958d83612f7a45b98093618b534f1308/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.882179 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a83dc6-1713-49b8-9701-fc44a5dd6d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e3068962a343d79494c5a846cf4b6a, user_id=958d83612f7a45b98093618b534f1308, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:48.882564 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-efc59cfd-4846-49b1-bc83-ccddace91ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3441131a52b40a0bca0dee5e7bb7c7f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:48.885214 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a83dc6-1713-49b8-9701-fc44a5dd6d1c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:48.889633 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-efc59cfd-4846-49b1-bc83-ccddace91ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3441131a52b40a0bca0dee5e7bb7c7f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:48.890340 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-efc59cfd-4846-49b1-bc83-ccddace91ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3441131a52b40a0bca0dee5e7bb7c7f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:48.890990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-efc59cfd-4846-49b1-bc83-ccddace91ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3441131a52b40a0bca0dee5e7bb7c7f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:48.898585 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-efc59cfd-4846-49b1-bc83-ccddace91ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3441131a52b40a0bca0dee5e7bb7c7f'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:48.899680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1378/2732] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:48 2022] PUT /identity/v3/projects/34e3068962a343d79494c5a846cf4b6a/users/958d83612f7a45b98093618b534f1308/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:48.903570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a83dc6-1713-49b8-9701-fc44a5dd6d1c admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:48.905346 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-29a83dc6-1713-49b8-9701-fc44a5dd6d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:48.917418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1355/2733] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:48 2022] DELETE /identity/v3/users/c3441131a52b40a0bca0dee5e7bb7c7f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:48.922153 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f31a93-0bfb-4da0-9e5c-955e956fd01c None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:48.922444 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f31a93-0bfb-4da0-9e5c-955e956fd01c None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:48.922724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f31a93-0bfb-4da0-9e5c-955e956fd01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:48.942006 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-93f31a93-0bfb-4da0-9e5c-955e956fd01c None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:48.943276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-93f31a93-0bfb-4da0-9e5c-955e956fd01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:48.948269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:48.975150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a83dc6-1713-49b8-9701-fc44a5dd6d1c admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:49.002408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:49.008075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1356/2734] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:49.018201 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f31a93-0bfb-4da0-9e5c-955e956fd01c None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:49.019584 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-93f31a93-0bfb-4da0-9e5c-955e956fd01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:49.036118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e214318-6cbc-4ae6-9464-8bb689ebe5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:49.036723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e214318-6cbc-4ae6-9464-8bb689ebe5fa admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:49.037001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e214318-6cbc-4ae6-9464-8bb689ebe5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:49.037309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e214318-6cbc-4ae6-9464-8bb689ebe5fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:49.037948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e214318-6cbc-4ae6-9464-8bb689ebe5fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:49.039592 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e214318-6cbc-4ae6-9464-8bb689ebe5fa admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:49.056288 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1379/2735] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:49.058394 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e214318-6cbc-4ae6-9464-8bb689ebe5fa admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:49.070791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e214318-6cbc-4ae6-9464-8bb689ebe5fa admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:49.076982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05759206-00b2-4618-b2d2-9ce0ef617be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:49.078372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05759206-00b2-4618-b2d2-9ce0ef617be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:49.078372 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05759206-00b2-4618-b2d2-9ce0ef617be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:49.078632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05759206-00b2-4618-b2d2-9ce0ef617be4 admin admin] PATH_INFO: `/v3/projects/34e3068962a343d79494c5a846cf4b6a/users/958d83612f7a45b98093618b534f1308/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:49.080663 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f31a93-0bfb-4da0-9e5c-955e956fd01c None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:49.081589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05759206-00b2-4618-b2d2-9ce0ef617be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e3068962a343d79494c5a846cf4b6a, user_id=958d83612f7a45b98093618b534f1308, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:49.084409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fca64-1587-420f-8968-46a6784146d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:49.084606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05759206-00b2-4618-b2d2-9ce0ef617be4 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:49.085317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fca64-1587-420f-8968-46a6784146d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:49.085939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fca64-1587-420f-8968-46a6784146d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:49.086166 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fca64-1587-420f-8968-46a6784146d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:49.096145 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fca64-1587-420f-8968-46a6784146d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:49.097748 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fca64-1587-420f-8968-46a6784146d8 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:49.097869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1380/2736] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:49 2022] PUT /identity/v3/projects/34e3068962a343d79494c5a846cf4b6a/users/958d83612f7a45b98093618b534f1308/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:49.108213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1357/2737] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:50:49.108816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d78b2d-d37c-455e-a9b6-978fbfbab838 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:49.109119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d78b2d-d37c-455e-a9b6-978fbfbab838 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:49.109414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d78b2d-d37c-455e-a9b6-978fbfbab838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:49.140307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b2d78b2d-d37c-455e-a9b6-978fbfbab838 None None] MFA Rules not processed for user `958d83612f7a45b98093618b534f1308`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:49.141982 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b2d78b2d-d37c-455e-a9b6-978fbfbab838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:49.218945 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1381/2738] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:49 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:49.231134 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fca64-1587-420f-8968-46a6784146d8 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:49.232736 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a22fca64-1587-420f-8968-46a6784146d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:49.295234 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fca64-1587-420f-8968-46a6784146d8 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:49.353019 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2cd5b-7549-4bcd-91aa-88cf36512e6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:49.353670 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2cd5b-7549-4bcd-91aa-88cf36512e6e service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:49.353946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2cd5b-7549-4bcd-91aa-88cf36512e6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:49.354213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2cd5b-7549-4bcd-91aa-88cf36512e6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:49.363420 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2cd5b-7549-4bcd-91aa-88cf36512e6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:49.364837 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2cd5b-7549-4bcd-91aa-88cf36512e6e service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:49.374863 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1358/2739] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:49 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:49.394628 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d78b2d-d37c-455e-a9b6-978fbfbab838 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:49.396155 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b2d78b2d-d37c-455e-a9b6-978fbfbab838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:49.463754 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2cd5b-7549-4bcd-91aa-88cf36512e6e service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:49.472762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2cd5b-7549-4bcd-91aa-88cf36512e6e service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:49.476342 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e6b91-ca61-433d-a991-f70aee4d4988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:49.476932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e6b91-ca61-433d-a991-f70aee4d4988 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:49.477252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e6b91-ca61-433d-a991-f70aee4d4988 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:49.477521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e6b91-ca61-433d-a991-f70aee4d4988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:49.479351 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-333e6b91-ca61-433d-a991-f70aee4d4988 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:49.483001 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d78b2d-d37c-455e-a9b6-978fbfbab838 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:49.487886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:49.488518 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:49.488810 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:49.489081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] PATH_INFO: `/v3/projects/17f582c9edbc4187a62e356226e53872` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:49.490408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17f582c9edbc4187a62e356226e53872)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:49.490538 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:49.492140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:49.537527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:49.546064 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e6b91-ca61-433d-a991-f70aee4d4988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:49.546708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17f582c9edbc4187a62e356226e53872', 'request_id': 'req-dae86515-7d35-4bf7-9825-13b5ef8652f4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:49.547528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e6b91-ca61-433d-a991-f70aee4d4988 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:49.553413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17f582c9edbc4187a62e356226e53872', 'request_id': 'req-dae86515-7d35-4bf7-9825-13b5ef8652f4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:49.556741 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1359/2740] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:49 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:50:49.560218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17f582c9edbc4187a62e356226e53872', 'request_id': 'req-dae86515-7d35-4bf7-9825-13b5ef8652f4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:49.574340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] The token cache is being invalidate because project 17f582c9edbc4187a62e356226e53872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:49.574710 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:49.575418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:49.576915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1382/2741] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:49 2022] DELETE /identity/v3/projects/17f582c9edbc4187a62e356226e53872 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:49.581681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e6b91-ca61-433d-a991-f70aee4d4988 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:49.593183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e6b91-ca61-433d-a991-f70aee4d4988 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:49.651011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef40e1-dbc0-4906-aa1f-a171307e3bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:49.651823 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef40e1-dbc0-4906-aa1f-a171307e3bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:49.652196 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef40e1-dbc0-4906-aa1f-a171307e3bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:49.652567 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef40e1-dbc0-4906-aa1f-a171307e3bc0 admin admin] PATH_INFO: `/v3/users/4fba4503c03c460881c008fdb50a3ea6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:49.653907 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef40e1-dbc0-4906-aa1f-a171307e3bc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fba4503c03c460881c008fdb50a3ea6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:49.656120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef40e1-dbc0-4906-aa1f-a171307e3bc0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:49.709416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-15ef40e1-dbc0-4906-aa1f-a171307e3bc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fba4503c03c460881c008fdb50a3ea6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:49.718002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-15ef40e1-dbc0-4906-aa1f-a171307e3bc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fba4503c03c460881c008fdb50a3ea6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:49.718794 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-15ef40e1-dbc0-4906-aa1f-a171307e3bc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fba4503c03c460881c008fdb50a3ea6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:49.719583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-15ef40e1-dbc0-4906-aa1f-a171307e3bc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fba4503c03c460881c008fdb50a3ea6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:49.727370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-15ef40e1-dbc0-4906-aa1f-a171307e3bc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fba4503c03c460881c008fdb50a3ea6'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:49.751255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1360/2742] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:49 2022] DELETE /identity/v3/users/4fba4503c03c460881c008fdb50a3ea6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:50.043153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:50.045084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:50.066269 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae86515-7d35-4bf7-9825-13b5ef8652f4 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:50.123726 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c7245-e39c-4765-bf74-72982df497f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:50.124364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c7245-e39c-4765-bf74-72982df497f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:50.124642 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c7245-e39c-4765-bf74-72982df497f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:50.124915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c7245-e39c-4765-bf74-72982df497f1 admin admin] PATH_INFO: `/v3/projects/772220a6b6744b05902864caa34d4a20` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:50.126063 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c7245-e39c-4765-bf74-72982df497f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=772220a6b6744b05902864caa34d4a20)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:50.128574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c7245-e39c-4765-bf74-72982df497f1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:50.175475 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-cf4c7245-e39c-4765-bf74-72982df497f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '772220a6b6744b05902864caa34d4a20', 'request_id': 'req-cf4c7245-e39c-4765-bf74-72982df497f1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:50.181025 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-cf4c7245-e39c-4765-bf74-72982df497f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '772220a6b6744b05902864caa34d4a20', 'request_id': 'req-cf4c7245-e39c-4765-bf74-72982df497f1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:50.187289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-cf4c7245-e39c-4765-bf74-72982df497f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '772220a6b6744b05902864caa34d4a20', 'request_id': 'req-cf4c7245-e39c-4765-bf74-72982df497f1'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:50.199721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-cf4c7245-e39c-4765-bf74-72982df497f1 admin admin] The token cache is being invalidate because project 772220a6b6744b05902864caa34d4a20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:50.200014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-cf4c7245-e39c-4765-bf74-72982df497f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:50.200632 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-cf4c7245-e39c-4765-bf74-72982df497f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:50.202209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1383/2743] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:50 2022] DELETE /identity/v3/projects/772220a6b6744b05902864caa34d4a20 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:50.207136 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61991452-1c3e-46a8-bf38-06e605fb7581 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:50.207421 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61991452-1c3e-46a8-bf38-06e605fb7581 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:50.207694 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61991452-1c3e-46a8-bf38-06e605fb7581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:50.227397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-61991452-1c3e-46a8-bf38-06e605fb7581 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:50.228597 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-61991452-1c3e-46a8-bf38-06e605fb7581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:50.257411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:50.293515 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1361/2744] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:50.357314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04407ae-0abf-4719-a064-d1afda0f7910 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:50.357597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04407ae-0abf-4719-a064-d1afda0f7910 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:50.357871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04407ae-0abf-4719-a064-d1afda0f7910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:50.378326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f04407ae-0abf-4719-a064-d1afda0f7910 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:50.379516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f04407ae-0abf-4719-a064-d1afda0f7910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:50.387216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1384/2745] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:50.391433 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61991452-1c3e-46a8-bf38-06e605fb7581 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:50.401057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61991452-1c3e-46a8-bf38-06e605fb7581 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:50.405911 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420e300-13a6-4dbf-9315-4558da60344d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:50.406506 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420e300-13a6-4dbf-9315-4558da60344d admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:50.406784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420e300-13a6-4dbf-9315-4558da60344d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:50.407060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420e300-13a6-4dbf-9315-4558da60344d admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:50.407701 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420e300-13a6-4dbf-9315-4558da60344d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:50:50.408524 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420e300-13a6-4dbf-9315-4558da60344d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:50.410144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420e300-13a6-4dbf-9315-4558da60344d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:50.421943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1362/2746] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:50:50.426823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04407ae-0abf-4719-a064-d1afda0f7910 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:50.435926 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04407ae-0abf-4719-a064-d1afda0f7910 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:50.440555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a861b0e-9710-4819-9fb0-4fb1e93042f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:50.441168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a861b0e-9710-4819-9fb0-4fb1e93042f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:50.441449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a861b0e-9710-4819-9fb0-4fb1e93042f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:50.441722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a861b0e-9710-4819-9fb0-4fb1e93042f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:50.442352 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a861b0e-9710-4819-9fb0-4fb1e93042f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:50.444250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a861b0e-9710-4819-9fb0-4fb1e93042f9 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:50.471776 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1385/2747] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:50 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:50.476047 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420e300-13a6-4dbf-9315-4558da60344d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:50.486955 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420e300-13a6-4dbf-9315-4558da60344d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:50.543141 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d84ff-b488-4ca2-9454-a0f26bc87577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:50.543737 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d84ff-b488-4ca2-9454-a0f26bc87577 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:50.544011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d84ff-b488-4ca2-9454-a0f26bc87577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:50.544276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d84ff-b488-4ca2-9454-a0f26bc87577 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:50.545213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d84ff-b488-4ca2-9454-a0f26bc87577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:50.547103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d84ff-b488-4ca2-9454-a0f26bc87577 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:50.568353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1363/2748] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:50 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:50:50.572463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a861b0e-9710-4819-9fb0-4fb1e93042f9 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:50.583595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a861b0e-9710-4819-9fb0-4fb1e93042f9 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:50.588433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e240d03-a0e7-4841-a1b3-24db7638888e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:50.589018 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e240d03-a0e7-4841-a1b3-24db7638888e admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:50.589328 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e240d03-a0e7-4841-a1b3-24db7638888e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:50.589601 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e240d03-a0e7-4841-a1b3-24db7638888e admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:50.590246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e240d03-a0e7-4841-a1b3-24db7638888e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:50.591853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e240d03-a0e7-4841-a1b3-24db7638888e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:50.603422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1386/2749] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:50.607529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d84ff-b488-4ca2-9454-a0f26bc87577 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:50.617396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d84ff-b488-4ca2-9454-a0f26bc87577 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:50.621989 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a3e75-3b60-4563-b465-8d190747435f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:50.622579 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a3e75-3b60-4563-b465-8d190747435f admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:50.622854 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a3e75-3b60-4563-b465-8d190747435f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:50.623123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a3e75-3b60-4563-b465-8d190747435f admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:50.623750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a3e75-3b60-4563-b465-8d190747435f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:50.625363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a3e75-3b60-4563-b465-8d190747435f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:50.635755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1364/2750] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:50.639779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e240d03-a0e7-4841-a1b3-24db7638888e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:50.650364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e240d03-a0e7-4841-a1b3-24db7638888e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:50.655384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25d7c1-14d6-4ab0-9912-7a0b7a8539b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:50.656036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25d7c1-14d6-4ab0-9912-7a0b7a8539b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:50.656333 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25d7c1-14d6-4ab0-9912-7a0b7a8539b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:50.656603 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25d7c1-14d6-4ab0-9912-7a0b7a8539b6 admin admin] PATH_INFO: `/v3/projects/82117a5000414ab8802f70b749b4dd33/users/4b9a57dc5fe1479ca5d7e25cc56f1da2/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:50.672455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25d7c1-14d6-4ab0-9912-7a0b7a8539b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82117a5000414ab8802f70b749b4dd33, user_id=4b9a57dc5fe1479ca5d7e25cc56f1da2, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:50.675114 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25d7c1-14d6-4ab0-9912-7a0b7a8539b6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:50.677272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:50.690136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1387/2751] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:50 2022] PUT /identity/v3/projects/82117a5000414ab8802f70b749b4dd33/users/4b9a57dc5fe1479ca5d7e25cc56f1da2/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:50.700267 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351e16e-607a-4637-b45d-9d8dbab8da40 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:50.700551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351e16e-607a-4637-b45d-9d8dbab8da40 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:50.700823 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351e16e-607a-4637-b45d-9d8dbab8da40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:50.728845 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-3351e16e-607a-4637-b45d-9d8dbab8da40 None None] MFA Rules not processed for user `4b9a57dc5fe1479ca5d7e25cc56f1da2`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:50.730092 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3351e16e-607a-4637-b45d-9d8dbab8da40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:50.795028 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1365/2752] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:50.799435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25d7c1-14d6-4ab0-9912-7a0b7a8539b6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:50.865497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25d7c1-14d6-4ab0-9912-7a0b7a8539b6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:50.871431 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000abf3b-3b91-4255-ae95-efda006e0d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:50.872093 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000abf3b-3b91-4255-ae95-efda006e0d2a admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:50.872354 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000abf3b-3b91-4255-ae95-efda006e0d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:50.872764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000abf3b-3b91-4255-ae95-efda006e0d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:50.873494 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000abf3b-3b91-4255-ae95-efda006e0d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:50.875388 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000abf3b-3b91-4255-ae95-efda006e0d2a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:50.907802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1388/2753] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:50 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:50.909777 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351e16e-607a-4637-b45d-9d8dbab8da40 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:50.923366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351e16e-607a-4637-b45d-9d8dbab8da40 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:50.949886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:50.987635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd87f88-ff99-4ebe-955f-24b2d89634d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:50.988241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd87f88-ff99-4ebe-955f-24b2d89634d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:50.988518 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd87f88-ff99-4ebe-955f-24b2d89634d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:50.988788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd87f88-ff99-4ebe-955f-24b2d89634d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:50.989441 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd87f88-ff99-4ebe-955f-24b2d89634d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:50.991337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd87f88-ff99-4ebe-955f-24b2d89634d2 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:51.011890 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1366/2754] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:50 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:51.016044 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000abf3b-3b91-4255-ae95-efda006e0d2a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:51.027178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000abf3b-3b91-4255-ae95-efda006e0d2a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:51.031884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65172183-16be-482a-9101-7e02e3afae77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:51.032473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65172183-16be-482a-9101-7e02e3afae77 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:51.032759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65172183-16be-482a-9101-7e02e3afae77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:51.033059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65172183-16be-482a-9101-7e02e3afae77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:51.033823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65172183-16be-482a-9101-7e02e3afae77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:51.035403 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65172183-16be-482a-9101-7e02e3afae77 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:51.046565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1389/2755] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:51.050850 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd87f88-ff99-4ebe-955f-24b2d89634d2 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:51.060860 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd87f88-ff99-4ebe-955f-24b2d89634d2 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:51.065992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94881d9-205f-4e16-b78a-2d6338df53cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:51.066522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94881d9-205f-4e16-b78a-2d6338df53cb admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:51.066792 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94881d9-205f-4e16-b78a-2d6338df53cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:51.067133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94881d9-205f-4e16-b78a-2d6338df53cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:51.067716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94881d9-205f-4e16-b78a-2d6338df53cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:51.069306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94881d9-205f-4e16-b78a-2d6338df53cb admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:51.078985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1367/2756] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:51.083917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65172183-16be-482a-9101-7e02e3afae77 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:51.093980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65172183-16be-482a-9101-7e02e3afae77 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:51.098577 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d92f31-d802-4a24-844a-8cdcf9bf0b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:51.099219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d92f31-d802-4a24-844a-8cdcf9bf0b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:51.099492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d92f31-d802-4a24-844a-8cdcf9bf0b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:51.099765 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d92f31-d802-4a24-844a-8cdcf9bf0b86 admin admin] PATH_INFO: `/v3/projects/81b9859b43774eb89728a493dc49e810/users/89492e90f511414f81858600154d101a/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:51.114064 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d92f31-d802-4a24-844a-8cdcf9bf0b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b9859b43774eb89728a493dc49e810, user_id=89492e90f511414f81858600154d101a, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:51.116717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d92f31-d802-4a24-844a-8cdcf9bf0b86 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:51.130255 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1390/2757] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:51 2022] PUT /identity/v3/projects/81b9859b43774eb89728a493dc49e810/users/89492e90f511414f81858600154d101a/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:51.140522 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f254bff-42dd-4c70-8460-78f590eb1b5f None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:51.140917 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f254bff-42dd-4c70-8460-78f590eb1b5f None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:51.141245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f254bff-42dd-4c70-8460-78f590eb1b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:51.171556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-4f254bff-42dd-4c70-8460-78f590eb1b5f None None] MFA Rules not processed for user `89492e90f511414f81858600154d101a`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:51.172765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4f254bff-42dd-4c70-8460-78f590eb1b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:51.241812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1368/2758] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:51.246261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d92f31-d802-4a24-844a-8cdcf9bf0b86 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:51.308329 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d92f31-d802-4a24-844a-8cdcf9bf0b86 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:51.313253 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d7b4b-f294-4dfd-a202-59c8f692d6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:51.313868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d7b4b-f294-4dfd-a202-59c8f692d6a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:51.314175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d7b4b-f294-4dfd-a202-59c8f692d6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:51.314557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d7b4b-f294-4dfd-a202-59c8f692d6a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:51.315682 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d7b4b-f294-4dfd-a202-59c8f692d6a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:51.317600 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d7b4b-f294-4dfd-a202-59c8f692d6a1 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:51.344360 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1391/2759] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:51 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:51.348596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f254bff-42dd-4c70-8460-78f590eb1b5f None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:51.359216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f254bff-42dd-4c70-8460-78f590eb1b5f None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:51.415256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb1e95-a4be-4a87-9541-82e8f5f76842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:51.415868 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb1e95-a4be-4a87-9541-82e8f5f76842 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:51.416175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb1e95-a4be-4a87-9541-82e8f5f76842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:51.416451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb1e95-a4be-4a87-9541-82e8f5f76842 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:51.417098 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb1e95-a4be-4a87-9541-82e8f5f76842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:51.419019 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb1e95-a4be-4a87-9541-82e8f5f76842 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:51.441750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1369/2760] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:51 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:51.445775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d7b4b-f294-4dfd-a202-59c8f692d6a1 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:51.456854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d7b4b-f294-4dfd-a202-59c8f692d6a1 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:51.461620 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230801a-c632-4fe2-a160-80d0f3401c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:51.462222 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230801a-c632-4fe2-a160-80d0f3401c2e admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:51.462505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230801a-c632-4fe2-a160-80d0f3401c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:51.462782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230801a-c632-4fe2-a160-80d0f3401c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:51.463422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230801a-c632-4fe2-a160-80d0f3401c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:51.464995 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230801a-c632-4fe2-a160-80d0f3401c2e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:51.475646 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1392/2761] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:51.479927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb1e95-a4be-4a87-9541-82e8f5f76842 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:51.490457 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb1e95-a4be-4a87-9541-82e8f5f76842 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:51.495037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0510f3-aaf2-41ab-aa4e-201f19930c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:51.495678 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0510f3-aaf2-41ab-aa4e-201f19930c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:51.495959 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0510f3-aaf2-41ab-aa4e-201f19930c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:51.496253 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0510f3-aaf2-41ab-aa4e-201f19930c54 admin admin] PATH_INFO: `/v3/projects/a08ec7bc87ca404484857951585cf6e6/users/2331e42178fb4fa38da6da0f466296a4/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:51.511406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0510f3-aaf2-41ab-aa4e-201f19930c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a08ec7bc87ca404484857951585cf6e6, user_id=2331e42178fb4fa38da6da0f466296a4, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:51.514135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0510f3-aaf2-41ab-aa4e-201f19930c54 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:51.526593 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:51.531557 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1370/2762] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:51 2022] PUT /identity/v3/projects/a08ec7bc87ca404484857951585cf6e6/users/2331e42178fb4fa38da6da0f466296a4/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:51.535881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230801a-c632-4fe2-a160-80d0f3401c2e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:51.546015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230801a-c632-4fe2-a160-80d0f3401c2e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:51.560251 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:51.607040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca2b79-8371-4f37-a00c-248e4a00b32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:51.607650 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca2b79-8371-4f37-a00c-248e4a00b32d admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:51.607934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca2b79-8371-4f37-a00c-248e4a00b32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:51.608206 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca2b79-8371-4f37-a00c-248e4a00b32d admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:51.608849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca2b79-8371-4f37-a00c-248e4a00b32d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:51.610476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca2b79-8371-4f37-a00c-248e4a00b32d admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:51.621418 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1393/2763] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:51.625526 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0510f3-aaf2-41ab-aa4e-201f19930c54 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:51.636524 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0510f3-aaf2-41ab-aa4e-201f19930c54 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:51.641111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b03d9b-e326-4749-bc2b-cd5766cd7fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:51.641765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b03d9b-e326-4749-bc2b-cd5766cd7fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:51.642046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b03d9b-e326-4749-bc2b-cd5766cd7fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:51.642349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b03d9b-e326-4749-bc2b-cd5766cd7fb4 admin admin] PATH_INFO: `/v3/projects/a08ec7bc87ca404484857951585cf6e6/users/2331e42178fb4fa38da6da0f466296a4/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:51.644256 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b03d9b-e326-4749-bc2b-cd5766cd7fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a08ec7bc87ca404484857951585cf6e6, user_id=2331e42178fb4fa38da6da0f466296a4, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:51.646981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b03d9b-e326-4749-bc2b-cd5766cd7fb4 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:51.657523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1371/2764] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:51 2022] PUT /identity/v3/projects/a08ec7bc87ca404484857951585cf6e6/users/2331e42178fb4fa38da6da0f466296a4/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:51.669361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e5954-67ae-47aa-be99-ca99d9ae5b82 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:51.669899 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e5954-67ae-47aa-be99-ca99d9ae5b82 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:51.670191 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e5954-67ae-47aa-be99-ca99d9ae5b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:51.699811 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-9b3e5954-67ae-47aa-be99-ca99d9ae5b82 None None] MFA Rules not processed for user `2331e42178fb4fa38da6da0f466296a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:51.701059 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9b3e5954-67ae-47aa-be99-ca99d9ae5b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:51.769851 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1394/2765] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:51.782171 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b03d9b-e326-4749-bc2b-cd5766cd7fb4 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:51.784375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e5b03d9b-e326-4749-bc2b-cd5766cd7fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:51.841886 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b03d9b-e326-4749-bc2b-cd5766cd7fb4 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:51.845952 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98aaff-51b9-4d51-bd81-251da9545df9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:51.846546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98aaff-51b9-4d51-bd81-251da9545df9 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:51.846821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98aaff-51b9-4d51-bd81-251da9545df9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:51.847087 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98aaff-51b9-4d51-bd81-251da9545df9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:51.902053 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98aaff-51b9-4d51-bd81-251da9545df9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:51.903481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98aaff-51b9-4d51-bd81-251da9545df9 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:51.912795 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1372/2766] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:54.496827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e5954-67ae-47aa-be99-ca99d9ae5b82 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:54.506680 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e5954-67ae-47aa-be99-ca99d9ae5b82 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:54.510209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70cc39-b775-44ac-a3e8-780fb2ff9150 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:54.510823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70cc39-b775-44ac-a3e8-780fb2ff9150 service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:54.511074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70cc39-b775-44ac-a3e8-780fb2ff9150 service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:54.511344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70cc39-b775-44ac-a3e8-780fb2ff9150 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:54.569777 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70cc39-b775-44ac-a3e8-780fb2ff9150 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:54.572215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70cc39-b775-44ac-a3e8-780fb2ff9150 service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:54.582754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1395/2767] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:54.843654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98aaff-51b9-4d51-bd81-251da9545df9 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:54.845551 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-3c98aaff-51b9-4d51-bd81-251da9545df9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:54.913995 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98aaff-51b9-4d51-bd81-251da9545df9 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:54.918673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:54.919921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:54.920207 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:54.920481 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] PATH_INFO: `/v3/users/044bb907ca2748f7a171dc0e1da1e0db` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:54.921646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=044bb907ca2748f7a171dc0e1da1e0db)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:54.923548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:54.971093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:54.984307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '044bb907ca2748f7a171dc0e1da1e0db'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:54.994577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '044bb907ca2748f7a171dc0e1da1e0db'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:54.996042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '044bb907ca2748f7a171dc0e1da1e0db'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:54.997106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '044bb907ca2748f7a171dc0e1da1e0db'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:55.006194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '044bb907ca2748f7a171dc0e1da1e0db'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:55.025601 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1373/2768] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:54 2022] DELETE /identity/v3/users/044bb907ca2748f7a171dc0e1da1e0db => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:55.030381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2e613-9af8-4d4b-bfbd-f919dc286d94 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:55.030675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2e613-9af8-4d4b-bfbd-f919dc286d94 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:55.031009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2e613-9af8-4d4b-bfbd-f919dc286d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:55.051980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-24c2e613-9af8-4d4b-bfbd-f919dc286d94 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:55.053302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-24c2e613-9af8-4d4b-bfbd-f919dc286d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:55.088341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:55.120572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1396/2769] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:55.131037 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:55.132656 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:55.202201 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49a798-c171-4b88-b423-80c98e98f6f4 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:55.206059 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8acfc9-3df8-48d7-8f1c-135e690540ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:55.206631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8acfc9-3df8-48d7-8f1c-135e690540ac service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:55.206986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8acfc9-3df8-48d7-8f1c-135e690540ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:55.207269 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8acfc9-3df8-48d7-8f1c-135e690540ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:55.217433 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2e613-9af8-4d4b-bfbd-f919dc286d94 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:55.217825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8acfc9-3df8-48d7-8f1c-135e690540ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:55.218978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-24c2e613-9af8-4d4b-bfbd-f919dc286d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:55.219366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8acfc9-3df8-48d7-8f1c-135e690540ac service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:55.229177 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1374/2770] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:55.282758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2e613-9af8-4d4b-bfbd-f919dc286d94 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:55.286570 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfb03c-7e19-4318-8165-821a9c7983ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:55.287175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfb03c-7e19-4318-8165-821a9c7983ff service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:55.287451 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfb03c-7e19-4318-8165-821a9c7983ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:55.287719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfb03c-7e19-4318-8165-821a9c7983ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:55.289510 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ecbfb03c-7e19-4318-8165-821a9c7983ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:55.359009 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfb03c-7e19-4318-8165-821a9c7983ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:55.360442 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfb03c-7e19-4318-8165-821a9c7983ff service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:55.370892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1397/2771] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:50:55.524156 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8acfc9-3df8-48d7-8f1c-135e690540ac service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:55.526244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ca8acfc9-3df8-48d7-8f1c-135e690540ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:55.535793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1856af-ef0f-4c9a-82f2-0d73e6fc2612 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:55.535793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1856af-ef0f-4c9a-82f2-0d73e6fc2612 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:55.535793 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1856af-ef0f-4c9a-82f2-0d73e6fc2612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:55.547303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8acfc9-3df8-48d7-8f1c-135e690540ac service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:55.554264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:55.554903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:55.555185 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:55.555482 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] PATH_INFO: `/v3/projects/a6423c6ffc56441a88fe59581fad6e9d` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:55.556616 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6423c6ffc56441a88fe59581fad6e9d)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:55.557504 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2f1856af-ef0f-4c9a-82f2-0d73e6fc2612 None None] MFA Rules not processed for user `e4ecba03d9254112b343f36ade20754f`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:55.558533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:55.558729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2f1856af-ef0f-4c9a-82f2-0d73e6fc2612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:55.613183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6423c6ffc56441a88fe59581fad6e9d', 'request_id': 'req-e1ee2bec-252c-4814-8377-0e4e20196f66'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:55.618732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6423c6ffc56441a88fe59581fad6e9d', 'request_id': 'req-e1ee2bec-252c-4814-8377-0e4e20196f66'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:55.620426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1398/2772] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:55.627165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6423c6ffc56441a88fe59581fad6e9d', 'request_id': 'req-e1ee2bec-252c-4814-8377-0e4e20196f66'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:55.639233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] The token cache is being invalidate because project a6423c6ffc56441a88fe59581fad6e9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:55.639552 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:55.640186 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:55.642023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1375/2773] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:55 2022] DELETE /identity/v3/projects/a6423c6ffc56441a88fe59581fad6e9d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:55.645975 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1856af-ef0f-4c9a-82f2-0d73e6fc2612 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:55.657489 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1856af-ef0f-4c9a-82f2-0d73e6fc2612 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:55.689817 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:55.718087 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:55.718734 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:55.719015 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:55.719291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] PATH_INFO: `/v3/users/958d83612f7a45b98093618b534f1308` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:55.720457 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] RBAC: Authorizing `identity:delete_user(user_id=958d83612f7a45b98093618b534f1308)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:55.723109 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:55.774962 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '958d83612f7a45b98093618b534f1308'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:55.788367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '958d83612f7a45b98093618b534f1308'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:55.789307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '958d83612f7a45b98093618b534f1308'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:55.797815 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '958d83612f7a45b98093618b534f1308'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:55.798537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '958d83612f7a45b98093618b534f1308'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:55.814627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1399/2774] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:55 2022] DELETE /identity/v3/users/958d83612f7a45b98093618b534f1308 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:55.983556 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:55.985285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:56.038336 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:56.046291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee2bec-252c-4814-8377-0e4e20196f66 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:56.099812 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:56.101947 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:56.105241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a83e38-79ce-4fe7-bcf9-afe90fd04c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:56.106265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a83e38-79ce-4fe7-bcf9-afe90fd04c07 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:56.107707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a83e38-79ce-4fe7-bcf9-afe90fd04c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:56.107707 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a83e38-79ce-4fe7-bcf9-afe90fd04c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:56.111000 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-66a83e38-79ce-4fe7-bcf9-afe90fd04c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:56.124986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a83e38-79ce-4fe7-bcf9-afe90fd04c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:56.127790 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a83e38-79ce-4fe7-bcf9-afe90fd04c07 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:56.144642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1376/2775] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:56.188186 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32416c6e-392a-4a2f-aae4-8ce9d520c63e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:56.194242 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4e5a6-1a9d-4879-92e7-c1973328c199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:56.195065 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4e5a6-1a9d-4879-92e7-c1973328c199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:56.195443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4e5a6-1a9d-4879-92e7-c1973328c199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:56.195853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4e5a6-1a9d-4879-92e7-c1973328c199 admin admin] PATH_INFO: `/v3/projects/34e3068962a343d79494c5a846cf4b6a` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:56.197077 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4e5a6-1a9d-4879-92e7-c1973328c199 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34e3068962a343d79494c5a846cf4b6a)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:56.199142 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4e5a6-1a9d-4879-92e7-c1973328c199 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:56.250866 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ecc4e5a6-1a9d-4879-92e7-c1973328c199 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34e3068962a343d79494c5a846cf4b6a', 'request_id': 'req-ecc4e5a6-1a9d-4879-92e7-c1973328c199'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:56.258723 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ecc4e5a6-1a9d-4879-92e7-c1973328c199 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34e3068962a343d79494c5a846cf4b6a', 'request_id': 'req-ecc4e5a6-1a9d-4879-92e7-c1973328c199'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:56.259822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:56.266756 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ecc4e5a6-1a9d-4879-92e7-c1973328c199 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34e3068962a343d79494c5a846cf4b6a', 'request_id': 'req-ecc4e5a6-1a9d-4879-92e7-c1973328c199'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:56.280515 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ecc4e5a6-1a9d-4879-92e7-c1973328c199 admin admin] The token cache is being invalidate because project 34e3068962a343d79494c5a846cf4b6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:50:56.280803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ecc4e5a6-1a9d-4879-92e7-c1973328c199 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:56.281456 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-ecc4e5a6-1a9d-4879-92e7-c1973328c199 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:56.282992 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1400/2776] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:50:56 2022] DELETE /identity/v3/projects/34e3068962a343d79494c5a846cf4b6a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:50:56.287813 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b73f4c-19ab-4f30-912d-a9fe6fe4fcd8 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:56.288093 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b73f4c-19ab-4f30-912d-a9fe6fe4fcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:56.288412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b73f4c-19ab-4f30-912d-a9fe6fe4fcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:56.308738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-b1b73f4c-19ab-4f30-912d-a9fe6fe4fcd8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:56.310110 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b1b73f4c-19ab-4f30-912d-a9fe6fe4fcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:56.376380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1377/2777] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:50:56.384620 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd17407-c262-461d-ac58-082471503412 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:56.384904 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd17407-c262-461d-ac58-082471503412 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:56.385208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd17407-c262-461d-ac58-082471503412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:56.405722 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-9dd17407-c262-461d-ac58-082471503412 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:56.406934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-9dd17407-c262-461d-ac58-082471503412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:56.417349 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1401/2778] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:56.419280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b73f4c-19ab-4f30-912d-a9fe6fe4fcd8 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:56.429123 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b73f4c-19ab-4f30-912d-a9fe6fe4fcd8 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:56.433789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1755a-2e6f-4fb1-b39b-ad9735af5a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:56.434399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1755a-2e6f-4fb1-b39b-ad9735af5a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:56.434692 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1755a-2e6f-4fb1-b39b-ad9735af5a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:56.435069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c1755a-2e6f-4fb1-b39b-ad9735af5a97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:56.435582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1755a-2e6f-4fb1-b39b-ad9735af5a97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:50:56.435898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1755a-2e6f-4fb1-b39b-ad9735af5a97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:56.437510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c1755a-2e6f-4fb1-b39b-ad9735af5a97 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:56.449878 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1378/2779] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:50:56.454406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd17407-c262-461d-ac58-082471503412 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:56.464573 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd17407-c262-461d-ac58-082471503412 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:56.469707 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b3d7e-8759-412d-af20-04c502f86ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:56.470304 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b3d7e-8759-412d-af20-04c502f86ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:56.470583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b3d7e-8759-412d-af20-04c502f86ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:56.470853 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b3d7e-8759-412d-af20-04c502f86ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:56.471477 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b3d7e-8759-412d-af20-04c502f86ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:56.473387 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b3d7e-8759-412d-af20-04c502f86ed5 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:56.507157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1402/2780] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:56 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:56.511798 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1755a-2e6f-4fb1-b39b-ad9735af5a97 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:56.523434 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c1755a-2e6f-4fb1-b39b-ad9735af5a97 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:56.592713 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c60401-e6f3-4c5f-99cc-f80338acce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:56.593516 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c60401-e6f3-4c5f-99cc-f80338acce59 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:56.593898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c60401-e6f3-4c5f-99cc-f80338acce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:56.594276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c60401-e6f3-4c5f-99cc-f80338acce59 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:56.595005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c60401-e6f3-4c5f-99cc-f80338acce59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:56.597052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c60401-e6f3-4c5f-99cc-f80338acce59 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:56.620461 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1379/2781] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:56 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:56.625126 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b3d7e-8759-412d-af20-04c502f86ed5 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:56.649780 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b3d7e-8759-412d-af20-04c502f86ed5 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:56.655870 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ed531-e438-4852-84dc-f280a14e7f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:56.657622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ed531-e438-4852-84dc-f280a14e7f62 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:56.660308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ed531-e438-4852-84dc-f280a14e7f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:56.661175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ed531-e438-4852-84dc-f280a14e7f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:56.662629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ed531-e438-4852-84dc-f280a14e7f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:56.665049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ed531-e438-4852-84dc-f280a14e7f62 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:56.678997 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c60401-e6f3-4c5f-99cc-f80338acce59 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:56.679649 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1403/2782] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:50:56.680946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-81c60401-e6f3-4c5f-99cc-f80338acce59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:56.683604 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ed531-e438-4852-84dc-f280a14e7f62 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:56.713306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:56.718660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ed531-e438-4852-84dc-f280a14e7f62 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:56.738746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca184cd6-25b7-42a8-b60b-31c41524ea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:56.739635 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca184cd6-25b7-42a8-b60b-31c41524ea2a admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:56.740225 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca184cd6-25b7-42a8-b60b-31c41524ea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:56.740697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca184cd6-25b7-42a8-b60b-31c41524ea2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:56.742258 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca184cd6-25b7-42a8-b60b-31c41524ea2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:56.744561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca184cd6-25b7-42a8-b60b-31c41524ea2a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:56.758030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1404/2783] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:56.762627 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca184cd6-25b7-42a8-b60b-31c41524ea2a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:56.769433 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c60401-e6f3-4c5f-99cc-f80338acce59 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:56.774062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca184cd6-25b7-42a8-b60b-31c41524ea2a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:56.775127 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff902029-969c-4779-a5f3-43edd8578998 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:56.775853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff902029-969c-4779-a5f3-43edd8578998 service placement] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:56.776142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff902029-969c-4779-a5f3-43edd8578998 service placement] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:56.776417 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff902029-969c-4779-a5f3-43edd8578998 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:56.778442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ff902029-969c-4779-a5f3-43edd8578998 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:56.782731 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330791f-ec1f-420b-98bb-03d4a827f89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:56.785171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330791f-ec1f-420b-98bb-03d4a827f89e admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:56.785584 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330791f-ec1f-420b-98bb-03d4a827f89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:56.786171 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330791f-ec1f-420b-98bb-03d4a827f89e admin admin] PATH_INFO: `/v3/projects/d78ff4bcc3b3461393c73d3ffd9144d3/users/f636fdecb2ad4548bb67dddf6210b2db/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:56.804874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff902029-969c-4779-a5f3-43edd8578998 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:56.806392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330791f-ec1f-420b-98bb-03d4a827f89e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78ff4bcc3b3461393c73d3ffd9144d3, user_id=f636fdecb2ad4548bb67dddf6210b2db, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:56.806667 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff902029-969c-4779-a5f3-43edd8578998 service placement] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:56.809057 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330791f-ec1f-420b-98bb-03d4a827f89e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:56.818993 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1380/2784] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:56.824412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1405/2785] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:56 2022] PUT /identity/v3/projects/d78ff4bcc3b3461393c73d3ffd9144d3/users/f636fdecb2ad4548bb67dddf6210b2db/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:56.834933 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26adc9-301d-47a4-9103-9bce4d12fec5 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:56.835672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26adc9-301d-47a4-9103-9bce4d12fec5 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:56.835672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26adc9-301d-47a4-9103-9bce4d12fec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:56.891089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-ca26adc9-301d-47a4-9103-9bce4d12fec5 None None] MFA Rules not processed for user `f636fdecb2ad4548bb67dddf6210b2db`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:56.892578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-ca26adc9-301d-47a4-9103-9bce4d12fec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:56.971052 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1381/2786] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:56.975737 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330791f-ec1f-420b-98bb-03d4a827f89e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:57.043504 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330791f-ec1f-420b-98bb-03d4a827f89e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:57.048474 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd38b0-42ca-4850-99d4-b730b3aa6862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:57.049083 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd38b0-42ca-4850-99d4-b730b3aa6862 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:57.049386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd38b0-42ca-4850-99d4-b730b3aa6862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:57.049670 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd38b0-42ca-4850-99d4-b730b3aa6862 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:57.050369 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd38b0-42ca-4850-99d4-b730b3aa6862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:57.052603 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cd38b0-42ca-4850-99d4-b730b3aa6862 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:57.081984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1406/2787] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:50:56 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:57.086832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26adc9-301d-47a4-9103-9bce4d12fec5 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:57.098032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26adc9-301d-47a4-9103-9bce4d12fec5 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:57.162547 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0b635-b60c-4c30-b216-86b06fc8ff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:57.163258 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0b635-b60c-4c30-b216-86b06fc8ff24 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:57.163536 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0b635-b60c-4c30-b216-86b06fc8ff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:57.163812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0b635-b60c-4c30-b216-86b06fc8ff24 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:57.164446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0b635-b60c-4c30-b216-86b06fc8ff24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:57.166380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0b635-b60c-4c30-b216-86b06fc8ff24 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:57.189683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1382/2788] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:50:57 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:50:57.194226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd38b0-42ca-4850-99d4-b730b3aa6862 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:57.205849 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd38b0-42ca-4850-99d4-b730b3aa6862 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:57.211687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97bd1e-fd1b-4419-b415-778193e88aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:57.212161 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97bd1e-fd1b-4419-b415-778193e88aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:57.212441 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97bd1e-fd1b-4419-b415-778193e88aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:57.212714 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97bd1e-fd1b-4419-b415-778193e88aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:57.213397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97bd1e-fd1b-4419-b415-778193e88aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:57.215039 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97bd1e-fd1b-4419-b415-778193e88aa0 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:57.226422 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1407/2789] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:57.230632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0b635-b60c-4c30-b216-86b06fc8ff24 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:57.241051 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0b635-b60c-4c30-b216-86b06fc8ff24 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:57.245713 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd6cb3-7e77-4eda-a364-e973b9b07b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:57.246349 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd6cb3-7e77-4eda-a364-e973b9b07b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:57.246622 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd6cb3-7e77-4eda-a364-e973b9b07b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:57.246919 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd6cb3-7e77-4eda-a364-e973b9b07b8a admin admin] PATH_INFO: `/v3/projects/530d131f7d9c47978ec4f40995d26051/users/72161906259d4f34bf8b4673e447fbce/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:57.264906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd6cb3-7e77-4eda-a364-e973b9b07b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=530d131f7d9c47978ec4f40995d26051, user_id=72161906259d4f34bf8b4673e447fbce, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:57.267594 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd6cb3-7e77-4eda-a364-e973b9b07b8a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:57.280016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1383/2790] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:57 2022] PUT /identity/v3/projects/530d131f7d9c47978ec4f40995d26051/users/72161906259d4f34bf8b4673e447fbce/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:57.284073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97bd1e-fd1b-4419-b415-778193e88aa0 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:57.294797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97bd1e-fd1b-4419-b415-778193e88aa0 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:57.337197 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:57.356839 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8a2f6-9f1b-49cc-863f-f6613dcdfc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:57.357558 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8a2f6-9f1b-49cc-863f-f6613dcdfc8a admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:57.357845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8a2f6-9f1b-49cc-863f-f6613dcdfc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:57.358118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8a2f6-9f1b-49cc-863f-f6613dcdfc8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:57.358834 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8a2f6-9f1b-49cc-863f-f6613dcdfc8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:57.360450 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8a2f6-9f1b-49cc-863f-f6613dcdfc8a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:57.371136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1408/2791] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:50:57.375439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd6cb3-7e77-4eda-a364-e973b9b07b8a admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:57.386903 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd6cb3-7e77-4eda-a364-e973b9b07b8a admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:57.391449 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc437c9-1187-43cc-aee2-82c8f51416b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:57.392143 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc437c9-1187-43cc-aee2-82c8f51416b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:57.392363 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc437c9-1187-43cc-aee2-82c8f51416b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:57.392625 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc437c9-1187-43cc-aee2-82c8f51416b0 admin admin] PATH_INFO: `/v3/projects/530d131f7d9c47978ec4f40995d26051/users/72161906259d4f34bf8b4673e447fbce/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:57.394582 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc437c9-1187-43cc-aee2-82c8f51416b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=530d131f7d9c47978ec4f40995d26051, user_id=72161906259d4f34bf8b4673e447fbce, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:57.397211 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc437c9-1187-43cc-aee2-82c8f51416b0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:57.399508 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:50:57.409268 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1384/2792] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:50:57 2022] PUT /identity/v3/projects/530d131f7d9c47978ec4f40995d26051/users/72161906259d4f34bf8b4673e447fbce/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:57.421248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bda67f-b117-4f77-bed1-867a4bf0b29d None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:57.421537 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bda67f-b117-4f77-bed1-867a4bf0b29d None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:57.421816 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bda67f-b117-4f77-bed1-867a4bf0b29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:57.452168 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f1bda67f-b117-4f77-bed1-867a4bf0b29d None None] MFA Rules not processed for user `72161906259d4f34bf8b4673e447fbce`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:57.453423 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f1bda67f-b117-4f77-bed1-867a4bf0b29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:57.545396 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1409/2793] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:57.554535 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc437c9-1187-43cc-aee2-82c8f51416b0 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:57.556682 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-dcc437c9-1187-43cc-aee2-82c8f51416b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:57.617245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc437c9-1187-43cc-aee2-82c8f51416b0 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:57.620897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffa02e-7849-4482-b6c1-6ee56eed57cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:57.621524 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffa02e-7849-4482-b6c1-6ee56eed57cd service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:57.621805 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffa02e-7849-4482-b6c1-6ee56eed57cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:57.622077 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffa02e-7849-4482-b6c1-6ee56eed57cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:57.631788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffa02e-7849-4482-b6c1-6ee56eed57cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:57.633285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffa02e-7849-4482-b6c1-6ee56eed57cd service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:57.642726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1385/2794] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:57.747022 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bda67f-b117-4f77-bed1-867a4bf0b29d None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:57.756988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bda67f-b117-4f77-bed1-867a4bf0b29d None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:57.760820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8229c3-3b17-453b-a1e4-265759cf9ccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:57.761498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8229c3-3b17-453b-a1e4-265759cf9ccb service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:57.761846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8229c3-3b17-453b-a1e4-265759cf9ccb service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:57.762183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8229c3-3b17-453b-a1e4-265759cf9ccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:57.826654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8229c3-3b17-453b-a1e4-265759cf9ccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:57.828456 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8229c3-3b17-453b-a1e4-265759cf9ccb service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:57.838556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1410/2795] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:50:59.616710 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffa02e-7849-4482-b6c1-6ee56eed57cd service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:59.618987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-abffa02e-7849-4482-b6c1-6ee56eed57cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:59.708069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffa02e-7849-4482-b6c1-6ee56eed57cd service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:50:59.714472 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:50:59.715416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:59.715937 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:59.716460 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] PATH_INFO: `/v3/users/4b9a57dc5fe1479ca5d7e25cc56f1da2` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:59.717992 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b9a57dc5fe1479ca5d7e25cc56f1da2)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:50:59.720075 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:50:59.786765 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b9a57dc5fe1479ca5d7e25cc56f1da2'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:59.796820 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b9a57dc5fe1479ca5d7e25cc56f1da2'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:59.797398 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b9a57dc5fe1479ca5d7e25cc56f1da2'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:59.798027 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b9a57dc5fe1479ca5d7e25cc56f1da2'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:59.807797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b9a57dc5fe1479ca5d7e25cc56f1da2'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:50:59.840984 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1386/2796] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:50:59 2022] DELETE /identity/v3/users/4b9a57dc5fe1479ca5d7e25cc56f1da2 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:50:59.842998 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6efad8-7726-4265-8cd4-92b0cf45d6ae None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:50:59.843479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6efad8-7726-4265-8cd4-92b0cf45d6ae None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:50:59.843619 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6efad8-7726-4265-8cd4-92b0cf45d6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:50:59.867306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2d6efad8-7726-4265-8cd4-92b0cf45d6ae None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:50:59.868879 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2d6efad8-7726-4265-8cd4-92b0cf45d6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:59.934158 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1411/2797] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:50:59.948688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:50:59.951058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:50:59.980749 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:00.030846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a0680-3a18-4e8c-9fe3-554394ab820f admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:00.034654 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8840719-61dc-499c-a4a5-d85d4e4e333d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:00.035259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8840719-61dc-499c-a4a5-d85d4e4e333d service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:00.035532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8840719-61dc-499c-a4a5-d85d4e4e333d service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:00.035804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8840719-61dc-499c-a4a5-d85d4e4e333d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:00.045524 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8840719-61dc-499c-a4a5-d85d4e4e333d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:00.046954 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8840719-61dc-499c-a4a5-d85d4e4e333d service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:00.056846 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1387/2798] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:50:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:51:00.504397 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6efad8-7726-4265-8cd4-92b0cf45d6ae None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:00.506115 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2d6efad8-7726-4265-8cd4-92b0cf45d6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:00.514307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:00.521321 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6efad8-7726-4265-8cd4-92b0cf45d6ae None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:00.527170 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:00.527689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:00.527974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:00.528252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] PATH_INFO: `/v3/projects/82117a5000414ab8802f70b749b4dd33` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:00.529443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] RBAC: Authorizing `identity:delete_project(project_id=82117a5000414ab8802f70b749b4dd33)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:00.531385 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:00.582443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82117a5000414ab8802f70b749b4dd33', 'request_id': 'req-781458fa-939a-40b1-9043-955d568b887e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.589172 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82117a5000414ab8802f70b749b4dd33', 'request_id': 'req-781458fa-939a-40b1-9043-955d568b887e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.596264 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82117a5000414ab8802f70b749b4dd33', 'request_id': 'req-781458fa-939a-40b1-9043-955d568b887e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.597828 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8840719-61dc-499c-a4a5-d85d4e4e333d service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:00.599370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a8840719-61dc-499c-a4a5-d85d4e4e333d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:00.610985 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] The token cache is being invalidate because project 82117a5000414ab8802f70b749b4dd33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:51:00.611281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.611913 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.613297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8840719-61dc-499c-a4a5-d85d4e4e333d service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:00.613956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1412/2799] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:51:00 2022] DELETE /identity/v3/projects/82117a5000414ab8802f70b749b4dd33 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:00.617721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:00.619505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:00.675721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d159b0a-24e5-44ff-a79d-f219b2396cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:00.676338 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d159b0a-24e5-44ff-a79d-f219b2396cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:00.676615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d159b0a-24e5-44ff-a79d-f219b2396cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:00.676885 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d159b0a-24e5-44ff-a79d-f219b2396cdc admin admin] PATH_INFO: `/v3/users/f636fdecb2ad4548bb67dddf6210b2db` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:00.678080 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d159b0a-24e5-44ff-a79d-f219b2396cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f636fdecb2ad4548bb67dddf6210b2db)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:00.679981 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d159b0a-24e5-44ff-a79d-f219b2396cdc admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:00.691336 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781458fa-939a-40b1-9043-955d568b887e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:00.697220 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:00.697861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:00.698144 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:00.698414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] PATH_INFO: `/v3/users/89492e90f511414f81858600154d101a` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:00.699574 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89492e90f511414f81858600154d101a)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:00.701505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:00.729821 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2d159b0a-24e5-44ff-a79d-f219b2396cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f636fdecb2ad4548bb67dddf6210b2db'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.738788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2d159b0a-24e5-44ff-a79d-f219b2396cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f636fdecb2ad4548bb67dddf6210b2db'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.739702 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2d159b0a-24e5-44ff-a79d-f219b2396cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f636fdecb2ad4548bb67dddf6210b2db'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.740387 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2d159b0a-24e5-44ff-a79d-f219b2396cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f636fdecb2ad4548bb67dddf6210b2db'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.748574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2d159b0a-24e5-44ff-a79d-f219b2396cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f636fdecb2ad4548bb67dddf6210b2db'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.755804 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89492e90f511414f81858600154d101a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.767993 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1388/2800] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:51:00 2022] DELETE /identity/v3/users/f636fdecb2ad4548bb67dddf6210b2db => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:51:00.771908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89492e90f511414f81858600154d101a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.772615 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd0908-fd3e-454e-b527-0aee359fca18 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:00.772740 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89492e90f511414f81858600154d101a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.772905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd0908-fd3e-454e-b527-0aee359fca18 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:00.773212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd0908-fd3e-454e-b527-0aee359fca18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:00.780216 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89492e90f511414f81858600154d101a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.781135 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89492e90f511414f81858600154d101a'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:00.793297 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-86cd0908-fd3e-454e-b527-0aee359fca18 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:00.794751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-86cd0908-fd3e-454e-b527-0aee359fca18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:00.796614 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1413/2801] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:51:00 2022] DELETE /identity/v3/users/89492e90f511414f81858600154d101a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:00.828165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:00.861665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1389/2802] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:51:00.872501 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:00.874323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:00.881697 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:00.941628 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea729f-6ade-4f1f-a644-eb96c5682404 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:00.945419 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45887e-cbfd-464c-94ec-c3f77a62d6a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:00.946085 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45887e-cbfd-464c-94ec-c3f77a62d6a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:00.946366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45887e-cbfd-464c-94ec-c3f77a62d6a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:00.946639 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45887e-cbfd-464c-94ec-c3f77a62d6a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:00.956465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45887e-cbfd-464c-94ec-c3f77a62d6a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:00.957920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45887e-cbfd-464c-94ec-c3f77a62d6a4 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:00.969002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1414/2803] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:51:01.095451 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd0908-fd3e-454e-b527-0aee359fca18 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:01.097506 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-86cd0908-fd3e-454e-b527-0aee359fca18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:01.118033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd0908-fd3e-454e-b527-0aee359fca18 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:01.124702 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:01.125753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:01.126218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:01.126525 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] PATH_INFO: `/v3/projects/81b9859b43774eb89728a493dc49e810` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:01.127739 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=81b9859b43774eb89728a493dc49e810)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:01.130171 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:01.186259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81b9859b43774eb89728a493dc49e810', 'request_id': 'req-e2444b8a-b040-4dd8-8f28-8207a302d1aa'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.192804 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81b9859b43774eb89728a493dc49e810', 'request_id': 'req-e2444b8a-b040-4dd8-8f28-8207a302d1aa'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.200605 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81b9859b43774eb89728a493dc49e810', 'request_id': 'req-e2444b8a-b040-4dd8-8f28-8207a302d1aa'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.214700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] The token cache is being invalidate because project 81b9859b43774eb89728a493dc49e810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:51:01.214986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.215629 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.217573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1390/2804] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:51:01 2022] DELETE /identity/v3/projects/81b9859b43774eb89728a493dc49e810 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:01.221204 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45887e-cbfd-464c-94ec-c3f77a62d6a4 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:01.231240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45887e-cbfd-464c-94ec-c3f77a62d6a4 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:01.291104 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:01.291764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:01.292040 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:01.292309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] PATH_INFO: `/v3/users/2331e42178fb4fa38da6da0f466296a4` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:01.293983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2331e42178fb4fa38da6da0f466296a4)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:01.296694 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:01.352724 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2331e42178fb4fa38da6da0f466296a4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.370807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2331e42178fb4fa38da6da0f466296a4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.371908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2331e42178fb4fa38da6da0f466296a4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.381565 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2331e42178fb4fa38da6da0f466296a4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.382586 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2331e42178fb4fa38da6da0f466296a4'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.398918 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1415/2805] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:51:01 2022] DELETE /identity/v3/users/2331e42178fb4fa38da6da0f466296a4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:01.431356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:01.432789 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:01.447031 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2444b8a-b040-4dd8-8f28-8207a302d1aa admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:01.523991 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:01.525375 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:01.525670 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:01.525950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] PATH_INFO: `/v3/projects/d78ff4bcc3b3461393c73d3ffd9144d3` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:01.528459 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d78ff4bcc3b3461393c73d3ffd9144d3)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:01.530194 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:01.609236 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd78ff4bcc3b3461393c73d3ffd9144d3', 'request_id': 'req-52abf9cb-e06c-448b-80ea-1434c7d68abe'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.619863 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd78ff4bcc3b3461393c73d3ffd9144d3', 'request_id': 'req-52abf9cb-e06c-448b-80ea-1434c7d68abe'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.623577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:01.629744 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd78ff4bcc3b3461393c73d3ffd9144d3', 'request_id': 'req-52abf9cb-e06c-448b-80ea-1434c7d68abe'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.649011 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] The token cache is being invalidate because project d78ff4bcc3b3461393c73d3ffd9144d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:51:01.649353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.650043 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.651595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1391/2806] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:51:01 2022] DELETE /identity/v3/projects/d78ff4bcc3b3461393c73d3ffd9144d3 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:01.655813 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:01.657425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:01.672208 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdc72d-b333-4101-94e4-945a3576557f admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:01.703911 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:01.741234 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed8b02-c36a-434b-beb2-b96b4f258d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:01.741869 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed8b02-c36a-434b-beb2-b96b4f258d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:01.742146 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed8b02-c36a-434b-beb2-b96b4f258d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:01.742416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed8b02-c36a-434b-beb2-b96b4f258d51 admin admin] PATH_INFO: `/v3/users/72161906259d4f34bf8b4673e447fbce` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:01.743557 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed8b02-c36a-434b-beb2-b96b4f258d51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72161906259d4f34bf8b4673e447fbce)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:01.745915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed8b02-c36a-434b-beb2-b96b4f258d51 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:01.806362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-37ed8b02-c36a-434b-beb2-b96b4f258d51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72161906259d4f34bf8b4673e447fbce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.820008 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-37ed8b02-c36a-434b-beb2-b96b4f258d51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72161906259d4f34bf8b4673e447fbce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.820771 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-37ed8b02-c36a-434b-beb2-b96b4f258d51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72161906259d4f34bf8b4673e447fbce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.828920 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-37ed8b02-c36a-434b-beb2-b96b4f258d51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72161906259d4f34bf8b4673e447fbce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.829674 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-37ed8b02-c36a-434b-beb2-b96b4f258d51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72161906259d4f34bf8b4673e447fbce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:01.847997 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1416/2807] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:51:01 2022] DELETE /identity/v3/users/72161906259d4f34bf8b4673e447fbce => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:01.885446 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:01.887371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:01.904382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abf9cb-e06c-448b-80ea-1434c7d68abe admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:01.989999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:01.990786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:01.991219 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:01.991510 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] PATH_INFO: `/v3/projects/a08ec7bc87ca404484857951585cf6e6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:01.993423 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a08ec7bc87ca404484857951585cf6e6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:01.996829 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.063814 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a08ec7bc87ca404484857951585cf6e6', 'request_id': 'req-d8480754-e17f-46de-939f-33fd5ad9a036'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:02.070880 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a08ec7bc87ca404484857951585cf6e6', 'request_id': 'req-d8480754-e17f-46de-939f-33fd5ad9a036'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:02.080335 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a08ec7bc87ca404484857951585cf6e6', 'request_id': 'req-d8480754-e17f-46de-939f-33fd5ad9a036'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:02.098383 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] The token cache is being invalidate because project a08ec7bc87ca404484857951585cf6e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:51:02.098883 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:02.099571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:02.101173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1392/2808] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:51:01 2022] DELETE /identity/v3/projects/a08ec7bc87ca404484857951585cf6e6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:51:02.106802 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19089999-9847-4b91-a744-ae39126016c1 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.107272 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19089999-9847-4b91-a744-ae39126016c1 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.107371 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19089999-9847-4b91-a744-ae39126016c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.134108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-19089999-9847-4b91-a744-ae39126016c1 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:02.135218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-19089999-9847-4b91-a744-ae39126016c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:02.194333 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.195733 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:02.202062 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1417/2809] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:51:02.209842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eadeaa-a2e1-4c7d-8399-179867e180fa None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.210132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eadeaa-a2e1-4c7d-8399-179867e180fa None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.210264 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8480754-e17f-46de-939f-33fd5ad9a036 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.210409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eadeaa-a2e1-4c7d-8399-179867e180fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.215159 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.215786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.216061 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.216343 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc admin admin] PATH_INFO: `/v3/projects/530d131f7d9c47978ec4f40995d26051` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.217628 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=530d131f7d9c47978ec4f40995d26051)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.219528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.231506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-e0eadeaa-a2e1-4c7d-8399-179867e180fa None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:02.232886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e0eadeaa-a2e1-4c7d-8399-179867e180fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:02.241556 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1418/2810] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:51:02.245664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eadeaa-a2e1-4c7d-8399-179867e180fa None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.256367 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eadeaa-a2e1-4c7d-8399-179867e180fa None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.262032 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ea75c-99a1-4379-869a-129473a23a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.262759 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ea75c-99a1-4379-869a-129473a23a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.263045 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ea75c-99a1-4379-869a-129473a23a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.263311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ea75c-99a1-4379-869a-129473a23a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.263978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ea75c-99a1-4379-869a-129473a23a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:51:02.264285 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ea75c-99a1-4379-869a-129473a23a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.265949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ea75c-99a1-4379-869a-129473a23a94 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.275353 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530d131f7d9c47978ec4f40995d26051', 'request_id': 'req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:02.279072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1419/2811] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:51:02.284509 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ea75c-99a1-4379-869a-129473a23a94 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.286089 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530d131f7d9c47978ec4f40995d26051', 'request_id': 'req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:02.297797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530d131f7d9c47978ec4f40995d26051', 'request_id': 'req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:02.300280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ea75c-99a1-4379-869a-129473a23a94 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.311384 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc admin admin] The token cache is being invalidate because project 530d131f7d9c47978ec4f40995d26051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:51:02.311682 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:02.312370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-2c9d60d1-1969-4e78-8b1b-3832ab53adcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:02.313908 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1393/2812] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:51:02 2022] DELETE /identity/v3/projects/530d131f7d9c47978ec4f40995d26051 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:02.318666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2684bc-65e9-425c-b708-df5128c9b1bd None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.319090 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2684bc-65e9-425c-b708-df5128c9b1bd None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.319407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2684bc-65e9-425c-b708-df5128c9b1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.327942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:02.329113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:02.340394 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-db2684bc-65e9-425c-b708-df5128c9b1bd None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:02.342362 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-db2684bc-65e9-425c-b708-df5128c9b1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:02.366698 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7280b-4e81-492e-87ac-3f24ee1789fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.367324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7280b-4e81-492e-87ac-3f24ee1789fa admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.367595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7280b-4e81-492e-87ac-3f24ee1789fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.367862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7280b-4e81-492e-87ac-3f24ee1789fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.368493 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7280b-4e81-492e-87ac-3f24ee1789fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.370399 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7280b-4e81-492e-87ac-3f24ee1789fa admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.403888 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1420/2813] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:51:02 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:51:02.404738 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7280b-4e81-492e-87ac-3f24ee1789fa admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.406309 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-59a7280b-4e81-492e-87ac-3f24ee1789fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:02.408200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1394/2814] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:51:02.416226 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb4177-66f3-4871-8824-091164db761d None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.416511 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb4177-66f3-4871-8824-091164db761d None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.416783 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb4177-66f3-4871-8824-091164db761d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.437386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-94eb4177-66f3-4871-8824-091164db761d None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:02.438571 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-94eb4177-66f3-4871-8824-091164db761d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:02.474400 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7280b-4e81-492e-87ac-3f24ee1789fa admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.483741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1a012-025c-45f3-87af-cd4766c57830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.484984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1a012-025c-45f3-87af-cd4766c57830 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.485320 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1a012-025c-45f3-87af-cd4766c57830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.485577 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1a012-025c-45f3-87af-cd4766c57830 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.486302 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1a012-025c-45f3-87af-cd4766c57830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.488767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1a012-025c-45f3-87af-cd4766c57830 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.502391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1395/2815] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:51:02.506709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb4177-66f3-4871-8824-091164db761d None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.511370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1421/2816] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:51:02 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:51:02.515436 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1a012-025c-45f3-87af-cd4766c57830 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.516174 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb4177-66f3-4871-8824-091164db761d None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.521751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c79e7-37eb-4052-b23a-dbf419541110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.522352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c79e7-37eb-4052-b23a-dbf419541110 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.522627 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c79e7-37eb-4052-b23a-dbf419541110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.522895 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c79e7-37eb-4052-b23a-dbf419541110 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.523662 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c79e7-37eb-4052-b23a-dbf419541110 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:51:02.523969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c79e7-37eb-4052-b23a-dbf419541110 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.525587 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c79e7-37eb-4052-b23a-dbf419541110 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.527821 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1a012-025c-45f3-87af-cd4766c57830 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.532585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e6091-0553-4dda-a81b-1a2ad36d99da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.533247 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e6091-0553-4dda-a81b-1a2ad36d99da admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.533527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e6091-0553-4dda-a81b-1a2ad36d99da admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.533803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e6091-0553-4dda-a81b-1a2ad36d99da admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.535189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e6091-0553-4dda-a81b-1a2ad36d99da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.536667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e6091-0553-4dda-a81b-1a2ad36d99da admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.541969 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1396/2817] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 08:51:02.542905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c79e7-37eb-4052-b23a-dbf419541110 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.550239 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1422/2818] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:51:02.551381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e6091-0553-4dda-a81b-1a2ad36d99da admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.552859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c79e7-37eb-4052-b23a-dbf419541110 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.558870 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd63bf4-b628-48ba-9291-89da0bc06a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.559493 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd63bf4-b628-48ba-9291-89da0bc06a09 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.559770 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd63bf4-b628-48ba-9291-89da0bc06a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.560042 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd63bf4-b628-48ba-9291-89da0bc06a09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.560667 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd63bf4-b628-48ba-9291-89da0bc06a09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.561435 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e6091-0553-4dda-a81b-1a2ad36d99da admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.562577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd63bf4-b628-48ba-9291-89da0bc06a09 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.566222 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e0755-7046-4900-a2b6-6156b86a7765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.566829 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e0755-7046-4900-a2b6-6156b86a7765 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.567307 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e0755-7046-4900-a2b6-6156b86a7765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.567589 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e0755-7046-4900-a2b6-6156b86a7765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.568232 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e0755-7046-4900-a2b6-6156b86a7765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.569917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e0755-7046-4900-a2b6-6156b86a7765 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.587084 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1423/2819] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:51:02.587718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e0755-7046-4900-a2b6-6156b86a7765 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.595087 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1397/2820] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:51:02 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:51:02.599241 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e0755-7046-4900-a2b6-6156b86a7765 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.599681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd63bf4-b628-48ba-9291-89da0bc06a09 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.664478 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7b880-b248-480e-ae54-40d9133e707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.665437 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7b880-b248-480e-ae54-40d9133e707f admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.665908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7b880-b248-480e-ae54-40d9133e707f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.666183 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7b880-b248-480e-ae54-40d9133e707f admin admin] PATH_INFO: `/v3/projects/63011f747824422da9f24e5aa43a57ad/users/ef7a012e1303456ea7abc94e96656199/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.673074 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd63bf4-b628-48ba-9291-89da0bc06a09 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.680862 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d263076-19d1-4288-b0d6-1200e0636007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.681943 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d263076-19d1-4288-b0d6-1200e0636007 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.682090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7b880-b248-480e-ae54-40d9133e707f admin admin] RBAC: Authorizing `identity:create_grant(project_id=63011f747824422da9f24e5aa43a57ad, user_id=ef7a012e1303456ea7abc94e96656199, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.682574 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d263076-19d1-4288-b0d6-1200e0636007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.683170 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d263076-19d1-4288-b0d6-1200e0636007 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.683988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d263076-19d1-4288-b0d6-1200e0636007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.684737 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7b880-b248-480e-ae54-40d9133e707f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.686700 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d263076-19d1-4288-b0d6-1200e0636007 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.700035 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1424/2821] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:51:02 2022] PUT /identity/v3/projects/63011f747824422da9f24e5aa43a57ad/users/ef7a012e1303456ea7abc94e96656199/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:02.705136 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b789-6053-44ea-a937-1cffc7668980 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.705484 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b789-6053-44ea-a937-1cffc7668980 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.706357 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b789-6053-44ea-a937-1cffc7668980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.709838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:02.713562 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:02.720950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1398/2822] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:51:02 2022] POST /identity/v3/users => generated 305 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:51:02.723018 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d263076-19d1-4288-b0d6-1200e0636007 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.727836 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5792b789-6053-44ea-a937-1cffc7668980 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:02.729044 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5792b789-6053-44ea-a937-1cffc7668980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:02.734352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d263076-19d1-4288-b0d6-1200e0636007 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.795453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a99e8-5d77-46c6-a5e4-3df2dc64ac19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.795783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1425/2823] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:51:02.796066 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a99e8-5d77-46c6-a5e4-3df2dc64ac19 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.796427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a99e8-5d77-46c6-a5e4-3df2dc64ac19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.796709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a99e8-5d77-46c6-a5e4-3df2dc64ac19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.797395 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a99e8-5d77-46c6-a5e4-3df2dc64ac19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.799318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a99e8-5d77-46c6-a5e4-3df2dc64ac19 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.805196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b789-6053-44ea-a937-1cffc7668980 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.806576 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5792b789-6053-44ea-a937-1cffc7668980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:02.812809 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1399/2824] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:51:02.814213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a99e8-5d77-46c6-a5e4-3df2dc64ac19 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.823818 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a99e8-5d77-46c6-a5e4-3df2dc64ac19 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.828483 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f8331-abb7-4e01-815c-2f5470958ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.829068 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f8331-abb7-4e01-815c-2f5470958ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.829376 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f8331-abb7-4e01-815c-2f5470958ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.829653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f8331-abb7-4e01-815c-2f5470958ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.830280 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f8331-abb7-4e01-815c-2f5470958ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.832418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f8331-abb7-4e01-815c-2f5470958ba0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.843271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1400/2825] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 08:51:02.847935 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f8331-abb7-4e01-815c-2f5470958ba0 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:02.857921 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f8331-abb7-4e01-815c-2f5470958ba0 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.862641 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0de90b-c9a4-43b3-870f-2862afdabdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.863291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0de90b-c9a4-43b3-870f-2862afdabdeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.863572 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0de90b-c9a4-43b3-870f-2862afdabdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.863848 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0de90b-c9a4-43b3-870f-2862afdabdeb admin admin] PATH_INFO: `/v3/projects/01405205da404f6497d76dd2ba2152b3/users/a04fe2061e0c48f5918fc3a37b1ec777/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.864631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b789-6053-44ea-a937-1cffc7668980 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:02.869264 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada76b76-9898-46db-89c0-ccf698fbab11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:02.869868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada76b76-9898-46db-89c0-ccf698fbab11 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.870164 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada76b76-9898-46db-89c0-ccf698fbab11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.870417 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada76b76-9898-46db-89c0-ccf698fbab11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.878909 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0de90b-c9a4-43b3-870f-2862afdabdeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=01405205da404f6497d76dd2ba2152b3, user_id=a04fe2061e0c48f5918fc3a37b1ec777, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.880637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada76b76-9898-46db-89c0-ccf698fbab11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:02.881609 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0de90b-c9a4-43b3-870f-2862afdabdeb admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.882099 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada76b76-9898-46db-89c0-ccf698fbab11 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:02.892762 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1426/2826] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:51:02.896033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1401/2827] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:51:02 2022] PUT /identity/v3/projects/01405205da404f6497d76dd2ba2152b3/users/a04fe2061e0c48f5918fc3a37b1ec777/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:51:02.906763 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e1836-45a0-4c4d-b411-342d2843c2ae None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:02.907058 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e1836-45a0-4c4d-b411-342d2843c2ae None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:02.907305 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e1836-45a0-4c4d-b411-342d2843c2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:02.939398 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-f45e1836-45a0-4c4d-b411-342d2843c2ae None None] MFA Rules not processed for user `a04fe2061e0c48f5918fc3a37b1ec777`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:02.941129 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-f45e1836-45a0-4c4d-b411-342d2843c2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:03.017196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1427/2828] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:02 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:51:03.022632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0de90b-c9a4-43b3-870f-2862afdabdeb admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:03.093296 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0de90b-c9a4-43b3-870f-2862afdabdeb admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:03.098120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30837947-c1b6-407f-b445-7a1081078147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:03.098755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30837947-c1b6-407f-b445-7a1081078147 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:03.099039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30837947-c1b6-407f-b445-7a1081078147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:03.099311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30837947-c1b6-407f-b445-7a1081078147 admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:03.099941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30837947-c1b6-407f-b445-7a1081078147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:03.101897 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30837947-c1b6-407f-b445-7a1081078147 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:03.133276 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1402/2829] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:51:03 2022] POST /identity/v3/projects => generated 409 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:51:03.134455 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e1836-45a0-4c4d-b411-342d2843c2ae None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:03.147610 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e1836-45a0-4c4d-b411-342d2843c2ae None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:03.181138 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:03.208354 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb893a42-002c-4450-b0ec-6b5b34479df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:03.209003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb893a42-002c-4450-b0ec-6b5b34479df7 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:03.209310 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb893a42-002c-4450-b0ec-6b5b34479df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:03.209583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb893a42-002c-4450-b0ec-6b5b34479df7 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:03.210240 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb893a42-002c-4450-b0ec-6b5b34479df7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:03.212173 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb893a42-002c-4450-b0ec-6b5b34479df7 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:03.240736 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1428/2830] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:51:03 2022] POST /identity/v3/users => generated 311 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:51:03.241927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30837947-c1b6-407f-b445-7a1081078147 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:03.253867 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30837947-c1b6-407f-b445-7a1081078147 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:03.259083 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19ecad-a161-4068-860c-51c15b2e47db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:03.259697 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19ecad-a161-4068-860c-51c15b2e47db admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:03.260009 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19ecad-a161-4068-860c-51c15b2e47db admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:03.260279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19ecad-a161-4068-860c-51c15b2e47db admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:03.260926 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc19ecad-a161-4068-860c-51c15b2e47db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:03.262642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc19ecad-a161-4068-860c-51c15b2e47db admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:03.278026 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1403/2831] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:51:03.279692 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb893a42-002c-4450-b0ec-6b5b34479df7 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:03.291361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb893a42-002c-4450-b0ec-6b5b34479df7 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:03.297333 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfb95a-3f0e-47ae-b30d-be871a2a1810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:03.298036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfb95a-3f0e-47ae-b30d-be871a2a1810 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:03.298856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfb95a-3f0e-47ae-b30d-be871a2a1810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:03.299152 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfb95a-3f0e-47ae-b30d-be871a2a1810 admin admin] PATH_INFO: `/v3/projects/ecfbd3841c0d4bd09af7afaa9a9b3816/users/6ad9ffbc2afd407aaf9777eb8599261c/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:03.321028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfb95a-3f0e-47ae-b30d-be871a2a1810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecfbd3841c0d4bd09af7afaa9a9b3816, user_id=6ad9ffbc2afd407aaf9777eb8599261c, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:03.323965 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfb95a-3f0e-47ae-b30d-be871a2a1810 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:03.337729 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1429/2832] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:51:03 2022] PUT /identity/v3/projects/ecfbd3841c0d4bd09af7afaa9a9b3816/users/6ad9ffbc2afd407aaf9777eb8599261c/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:03.341665 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19ecad-a161-4068-860c-51c15b2e47db admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:03.354072 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19ecad-a161-4068-860c-51c15b2e47db admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:03.388830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:03.415422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a1230-bab3-4620-9db0-5be2a7306dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:03.416114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a1230-bab3-4620-9db0-5be2a7306dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:03.416391 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a1230-bab3-4620-9db0-5be2a7306dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:03.416664 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a1230-bab3-4620-9db0-5be2a7306dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:03.417339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a1230-bab3-4620-9db0-5be2a7306dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:03.418925 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a1230-bab3-4620-9db0-5be2a7306dbf admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:03.429975 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1404/2833] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:51:03.434406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfb95a-3f0e-47ae-b30d-be871a2a1810 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:03.445770 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfb95a-3f0e-47ae-b30d-be871a2a1810 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:03.450784 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb95b0-f02d-4631-95ed-6a388f1007be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:03.451519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb95b0-f02d-4631-95ed-6a388f1007be admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:03.451857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb95b0-f02d-4631-95ed-6a388f1007be admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:03.452215 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb95b0-f02d-4631-95ed-6a388f1007be admin admin] PATH_INFO: `/v3/projects/ecfbd3841c0d4bd09af7afaa9a9b3816/users/6ad9ffbc2afd407aaf9777eb8599261c/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:03.454210 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb95b0-f02d-4631-95ed-6a388f1007be admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecfbd3841c0d4bd09af7afaa9a9b3816, user_id=6ad9ffbc2afd407aaf9777eb8599261c, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:03.456917 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb95b0-f02d-4631-95ed-6a388f1007be admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:03.467755 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1430/2834] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:51:03 2022] PUT /identity/v3/projects/ecfbd3841c0d4bd09af7afaa9a9b3816/users/6ad9ffbc2afd407aaf9777eb8599261c/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:03.479371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0537fc9-f6d0-4896-8229-24d40f56b738 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:03.479718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0537fc9-f6d0-4896-8229-24d40f56b738 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:03.480368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0537fc9-f6d0-4896-8229-24d40f56b738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:03.510050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-c0537fc9-f6d0-4896-8229-24d40f56b738 None None] MFA Rules not processed for user `6ad9ffbc2afd407aaf9777eb8599261c`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:03.511346 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c0537fc9-f6d0-4896-8229-24d40f56b738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:03.582748 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1754c2-fe27-46cd-9c5a-55415fa12fd7 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:03.583153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1754c2-fe27-46cd-9c5a-55415fa12fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:03.583530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1754c2-fe27-46cd-9c5a-55415fa12fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:03.591905 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1405/2835] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:03 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:51:03.602787 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0537fc9-f6d0-4896-8229-24d40f56b738 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:03.604183 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c0537fc9-f6d0-4896-8229-24d40f56b738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:03.617412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-ef1754c2-fe27-46cd-9c5a-55415fa12fd7 None None] MFA Rules not processed for user `ef7a012e1303456ea7abc94e96656199`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:03.618760 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ef1754c2-fe27-46cd-9c5a-55415fa12fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:03.672742 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0537fc9-f6d0-4896-8229-24d40f56b738 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:03.676999 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29c1cc-a94c-4a0e-9029-4ea80b974833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:03.677653 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29c1cc-a94c-4a0e-9029-4ea80b974833 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:03.677928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29c1cc-a94c-4a0e-9029-4ea80b974833 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:03.678197 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29c1cc-a94c-4a0e-9029-4ea80b974833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:03.689214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29c1cc-a94c-4a0e-9029-4ea80b974833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:03.691058 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29c1cc-a94c-4a0e-9029-4ea80b974833 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:03.702468 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1431/2836] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:51:03.711030 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1754c2-fe27-46cd-9c5a-55415fa12fd7 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:03.711475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1406/2837] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:03 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:51:03.712391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-ef1754c2-fe27-46cd-9c5a-55415fa12fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:03.726868 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1754c2-fe27-46cd-9c5a-55415fa12fd7 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:03.731490 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a3f77-cf66-46ee-abb7-d0cc8295fa9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:03.732155 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a3f77-cf66-46ee-abb7-d0cc8295fa9d service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:03.732441 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a3f77-cf66-46ee-abb7-d0cc8295fa9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:03.733143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a3f77-cf66-46ee-abb7-d0cc8295fa9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:03.743278 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a3f77-cf66-46ee-abb7-d0cc8295fa9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:03.745167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a3f77-cf66-46ee-abb7-d0cc8295fa9d service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:03.765819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1432/2838] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:51:03.918755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29c1cc-a94c-4a0e-9029-4ea80b974833 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:03.937200 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29c1cc-a94c-4a0e-9029-4ea80b974833 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:03.943784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14503326-c8ec-4dd9-bfbd-dc085e3d9b47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:03.944509 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14503326-c8ec-4dd9-bfbd-dc085e3d9b47 service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:03.945002 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14503326-c8ec-4dd9-bfbd-dc085e3d9b47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:03.945400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14503326-c8ec-4dd9-bfbd-dc085e3d9b47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:04.014023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14503326-c8ec-4dd9-bfbd-dc085e3d9b47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:04.015728 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14503326-c8ec-4dd9-bfbd-dc085e3d9b47 service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:04.020050 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:04.026800 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1407/2839] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:03 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:51:05.180854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4698807-4d23-4fb1-abab-9607f1bf15d1 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:05.181443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4698807-4d23-4fb1-abab-9607f1bf15d1 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:05.181548 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4698807-4d23-4fb1-abab-9607f1bf15d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:05.200687 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-b4698807-4d23-4fb1-abab-9607f1bf15d1 None None] MFA Rules not processed for user `ef7a012e1303456ea7abc94e96656199`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:05.201950 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-b4698807-4d23-4fb1-abab-9607f1bf15d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:05.208664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1433/2840] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:51:13.442529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14503326-c8ec-4dd9-bfbd-dc085e3d9b47 service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:13.444121 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-14503326-c8ec-4dd9-bfbd-dc085e3d9b47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:13.528301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4698807-4d23-4fb1-abab-9607f1bf15d1 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:13.532655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14503326-c8ec-4dd9-bfbd-dc085e3d9b47 service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:13.538902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec46659-6cb1-4248-a25c-68688148d747 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:13.539503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec46659-6cb1-4248-a25c-68688148d747 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:13.539781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec46659-6cb1-4248-a25c-68688148d747 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:13.540048 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec46659-6cb1-4248-a25c-68688148d747 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:13.541887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-4ec46659-6cb1-4248-a25c-68688148d747 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:13.560301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec46659-6cb1-4248-a25c-68688148d747 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:13.563056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec46659-6cb1-4248-a25c-68688148d747 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:13.574657 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1408/2841] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:51:13.603706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:13.609137 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4698807-4d23-4fb1-abab-9607f1bf15d1 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:13.615696 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:13.616342 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:13.616657 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:13.616878 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] PATH_INFO: `/v3/users/a04fe2061e0c48f5918fc3a37b1ec777` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:13.618052 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a04fe2061e0c48f5918fc3a37b1ec777)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:13.619948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:13.669718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a04fe2061e0c48f5918fc3a37b1ec777'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:13.683757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a04fe2061e0c48f5918fc3a37b1ec777'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:13.684498 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a04fe2061e0c48f5918fc3a37b1ec777'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:13.692924 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a04fe2061e0c48f5918fc3a37b1ec777'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:13.693676 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a04fe2061e0c48f5918fc3a37b1ec777'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:13.711558 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1434/2842] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:51:13 2022] DELETE /identity/v3/users/a04fe2061e0c48f5918fc3a37b1ec777 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:13.713142 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa59f3-adcd-4e2b-99a5-98a72fde806b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:13.713433 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa59f3-adcd-4e2b-99a5-98a72fde806b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:13.713706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa59f3-adcd-4e2b-99a5-98a72fde806b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:13.734240 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-7bfa59f3-adcd-4e2b-99a5-98a72fde806b None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:13.735449 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7bfa59f3-adcd-4e2b-99a5-98a72fde806b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:13.798406 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1409/2843] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:51:13.808865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:13.810409 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:13.868102 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ef1ee-05f8-4de4-9095-2ab91ad244c6 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:13.871757 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d2bed-c26b-4140-b232-a272ad34e64f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:13.872358 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d2bed-c26b-4140-b232-a272ad34e64f service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:13.872637 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d2bed-c26b-4140-b232-a272ad34e64f service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:13.872907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d2bed-c26b-4140-b232-a272ad34e64f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:13.882155 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d2bed-c26b-4140-b232-a272ad34e64f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:13.883585 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d2bed-c26b-4140-b232-a272ad34e64f service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:13.893625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1435/2844] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:51:14.155285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa59f3-adcd-4e2b-99a5-98a72fde806b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:14.156825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7bfa59f3-adcd-4e2b-99a5-98a72fde806b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:14.169111 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa59f3-adcd-4e2b-99a5-98a72fde806b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:14.174103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:14.174735 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:14.175029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:14.175290 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] PATH_INFO: `/v3/projects/01405205da404f6497d76dd2ba2152b3` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:14.176424 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01405205da404f6497d76dd2ba2152b3)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:14.178386 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:14.226158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01405205da404f6497d76dd2ba2152b3', 'request_id': 'req-7680d909-0ff6-4506-8a86-b05b671a2bc9'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.231588 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01405205da404f6497d76dd2ba2152b3', 'request_id': 'req-7680d909-0ff6-4506-8a86-b05b671a2bc9'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.239370 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01405205da404f6497d76dd2ba2152b3', 'request_id': 'req-7680d909-0ff6-4506-8a86-b05b671a2bc9'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.253755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] The token cache is being invalidate because project 01405205da404f6497d76dd2ba2152b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:51:14.254048 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.254731 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.256271 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1410/2845] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:51:14 2022] DELETE /identity/v3/projects/01405205da404f6497d76dd2ba2152b3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:14.260268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d2bed-c26b-4140-b232-a272ad34e64f service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:14.270648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d2bed-c26b-4140-b232-a272ad34e64f service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:14.328514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c30e6a-9a49-4c0a-a26b-a709dc0fc123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:14.329199 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c30e6a-9a49-4c0a-a26b-a709dc0fc123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:14.329477 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c30e6a-9a49-4c0a-a26b-a709dc0fc123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:14.329745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c30e6a-9a49-4c0a-a26b-a709dc0fc123 admin admin] PATH_INFO: `/v3/users/6ad9ffbc2afd407aaf9777eb8599261c` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:14.331507 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c30e6a-9a49-4c0a-a26b-a709dc0fc123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ad9ffbc2afd407aaf9777eb8599261c)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:14.333438 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c30e6a-9a49-4c0a-a26b-a709dc0fc123 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:14.358054 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:14.408291 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b3c30e6a-9a49-4c0a-a26b-a709dc0fc123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ad9ffbc2afd407aaf9777eb8599261c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.424847 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b3c30e6a-9a49-4c0a-a26b-a709dc0fc123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ad9ffbc2afd407aaf9777eb8599261c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.425587 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b3c30e6a-9a49-4c0a-a26b-a709dc0fc123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ad9ffbc2afd407aaf9777eb8599261c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.433315 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b3c30e6a-9a49-4c0a-a26b-a709dc0fc123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ad9ffbc2afd407aaf9777eb8599261c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.434004 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-b3c30e6a-9a49-4c0a-a26b-a709dc0fc123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ad9ffbc2afd407aaf9777eb8599261c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.448351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1436/2846] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:51:14 2022] DELETE /identity/v3/users/6ad9ffbc2afd407aaf9777eb8599261c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:14.741265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:14.742864 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:14.766670 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680d909-0ff6-4506-8a86-b05b671a2bc9 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:14.772192 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:14.832352 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd1116-aa62-4193-a048-fcd630a7d8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:14.833001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd1116-aa62-4193-a048-fcd630a7d8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:14.833316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd1116-aa62-4193-a048-fcd630a7d8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:14.833593 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd1116-aa62-4193-a048-fcd630a7d8eb admin admin] PATH_INFO: `/v3/projects/ecfbd3841c0d4bd09af7afaa9a9b3816` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:14.834752 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd1116-aa62-4193-a048-fcd630a7d8eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecfbd3841c0d4bd09af7afaa9a9b3816)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:14.836725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd1116-aa62-4193-a048-fcd630a7d8eb admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:14.886835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-38fd1116-aa62-4193-a048-fcd630a7d8eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecfbd3841c0d4bd09af7afaa9a9b3816', 'request_id': 'req-38fd1116-aa62-4193-a048-fcd630a7d8eb'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.893001 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-38fd1116-aa62-4193-a048-fcd630a7d8eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecfbd3841c0d4bd09af7afaa9a9b3816', 'request_id': 'req-38fd1116-aa62-4193-a048-fcd630a7d8eb'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.901442 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-38fd1116-aa62-4193-a048-fcd630a7d8eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecfbd3841c0d4bd09af7afaa9a9b3816', 'request_id': 'req-38fd1116-aa62-4193-a048-fcd630a7d8eb'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.915195 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-38fd1116-aa62-4193-a048-fcd630a7d8eb admin admin] The token cache is being invalidate because project ecfbd3841c0d4bd09af7afaa9a9b3816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:51:14.915488 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-38fd1116-aa62-4193-a048-fcd630a7d8eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.916141 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-38fd1116-aa62-4193-a048-fcd630a7d8eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:14.917626 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1411/2847] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:51:14 2022] DELETE /identity/v3/projects/ecfbd3841c0d4bd09af7afaa9a9b3816 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:14.922791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457978a0-b759-4f9f-99ba-5823bd72c599 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:14.923277 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457978a0-b759-4f9f-99ba-5823bd72c599 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:14.923393 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457978a0-b759-4f9f-99ba-5823bd72c599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:14.943972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-457978a0-b759-4f9f-99ba-5823bd72c599 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:14.945217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-457978a0-b759-4f9f-99ba-5823bd72c599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:15.011344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1437/2848] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:51:15.020109 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f05dc-5a09-4ec3-bfa7-802b6e8138a3 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:15.020517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f05dc-5a09-4ec3-bfa7-802b6e8138a3 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:15.020880 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f05dc-5a09-4ec3-bfa7-802b6e8138a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:15.041532 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-d22f05dc-5a09-4ec3-bfa7-802b6e8138a3 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:15.042755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d22f05dc-5a09-4ec3-bfa7-802b6e8138a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:15.051106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1412/2849] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:51:15.056930 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457978a0-b759-4f9f-99ba-5823bd72c599 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:15.070737 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457978a0-b759-4f9f-99ba-5823bd72c599 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:15.077938 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f217ce-fb9a-4698-b803-5b75816aa626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:15.078745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f217ce-fb9a-4698-b803-5b75816aa626 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:15.079049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f217ce-fb9a-4698-b803-5b75816aa626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:15.079348 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f217ce-fb9a-4698-b803-5b75816aa626 admin admin] PATH_INFO: `/v3/domains` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:15.079980 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f217ce-fb9a-4698-b803-5b75816aa626 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:51:15.080289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f217ce-fb9a-4698-b803-5b75816aa626 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:15.082750 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f217ce-fb9a-4698-b803-5b75816aa626 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:15.098483 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1438/2850] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:51:15.103059 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f05dc-5a09-4ec3-bfa7-802b6e8138a3 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:15.115746 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f05dc-5a09-4ec3-bfa7-802b6e8138a3 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:15.120411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f77a4b-fa42-4689-8301-e30966fcaf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:15.121003 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f77a4b-fa42-4689-8301-e30966fcaf8b admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:15.121316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f77a4b-fa42-4689-8301-e30966fcaf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:15.121586 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f77a4b-fa42-4689-8301-e30966fcaf8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:15.122214 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f77a4b-fa42-4689-8301-e30966fcaf8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:15.124103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f77a4b-fa42-4689-8301-e30966fcaf8b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:15.152675 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1413/2851] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:51:15 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:51:15.157155 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f217ce-fb9a-4698-b803-5b75816aa626 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:15.168797 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f217ce-fb9a-4698-b803-5b75816aa626 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:15.239425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe8835-72aa-4ebc-a22d-59dcabf1dd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:15.240049 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe8835-72aa-4ebc-a22d-59dcabf1dd89 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:15.240331 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe8835-72aa-4ebc-a22d-59dcabf1dd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:15.240606 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe8835-72aa-4ebc-a22d-59dcabf1dd89 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:15.241280 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe8835-72aa-4ebc-a22d-59dcabf1dd89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:15.243218 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe8835-72aa-4ebc-a22d-59dcabf1dd89 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:15.267465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1439/2852] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:51:15 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:51:15.272822 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f77a4b-fa42-4689-8301-e30966fcaf8b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:15.283916 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f77a4b-fa42-4689-8301-e30966fcaf8b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:15.289412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d18e9-11aa-47c3-85e0-4bfb25afb84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:15.290535 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d18e9-11aa-47c3-85e0-4bfb25afb84c admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:15.291019 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d18e9-11aa-47c3-85e0-4bfb25afb84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:15.291291 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d18e9-11aa-47c3-85e0-4bfb25afb84c admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:15.291946 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d18e9-11aa-47c3-85e0-4bfb25afb84c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:15.295438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d18e9-11aa-47c3-85e0-4bfb25afb84c admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:15.306048 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1414/2853] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:51:15.310196 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe8835-72aa-4ebc-a22d-59dcabf1dd89 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:15.322527 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe8835-72aa-4ebc-a22d-59dcabf1dd89 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:15.328529 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930de624-6073-4e03-8f0b-6212ba5bcb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:15.329201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930de624-6073-4e03-8f0b-6212ba5bcb6c admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:15.329485 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930de624-6073-4e03-8f0b-6212ba5bcb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:15.329758 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930de624-6073-4e03-8f0b-6212ba5bcb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:15.330434 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930de624-6073-4e03-8f0b-6212ba5bcb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:15.332047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930de624-6073-4e03-8f0b-6212ba5bcb6c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:15.345067 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1440/2854] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:51:15.351337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d18e9-11aa-47c3-85e0-4bfb25afb84c admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:15.363397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d18e9-11aa-47c3-85e0-4bfb25afb84c admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:15.368655 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62668c-76ba-4944-99ac-f2e239ae62bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:15.369646 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62668c-76ba-4944-99ac-f2e239ae62bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:15.369925 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62668c-76ba-4944-99ac-f2e239ae62bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:15.370201 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62668c-76ba-4944-99ac-f2e239ae62bf admin admin] PATH_INFO: `/v3/projects/f110f673508741b3aa40aa8e7a1d33e1/users/577947ea9d964803b6db65f0f84d87f5/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:15.387537 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62668c-76ba-4944-99ac-f2e239ae62bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f110f673508741b3aa40aa8e7a1d33e1, user_id=577947ea9d964803b6db65f0f84d87f5, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:15.390309 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62668c-76ba-4944-99ac-f2e239ae62bf admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:15.419133 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1415/2855] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:51:15 2022] PUT /identity/v3/projects/f110f673508741b3aa40aa8e7a1d33e1/users/577947ea9d964803b6db65f0f84d87f5/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:15.422228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7aeac2-fbbe-4908-94ee-22ff8841e11b None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:15.422521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7aeac2-fbbe-4908-94ee-22ff8841e11b None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:15.422800 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7aeac2-fbbe-4908-94ee-22ff8841e11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:15.432915 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:15.459561 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5b7aeac2-fbbe-4908-94ee-22ff8841e11b None None] MFA Rules not processed for user `577947ea9d964803b6db65f0f84d87f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:15.460854 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5b7aeac2-fbbe-4908-94ee-22ff8841e11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:15.548007 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1441/2856] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:51:15.549726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62668c-76ba-4944-99ac-f2e239ae62bf admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:15.591489 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:15.629725 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62668c-76ba-4944-99ac-f2e239ae62bf admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:15.635674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc23bd9-fa75-43c1-94b1-3010e11ba26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:15.636305 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc23bd9-fa75-43c1-94b1-3010e11ba26c admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:15.636585 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc23bd9-fa75-43c1-94b1-3010e11ba26c admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:15.636857 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc23bd9-fa75-43c1-94b1-3010e11ba26c admin admin] PATH_INFO: `/v3/projects` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:15.637540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc23bd9-fa75-43c1-94b1-3010e11ba26c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:15.639476 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc23bd9-fa75-43c1-94b1-3010e11ba26c admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:15.671337 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1416/2857] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:51:15 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:51:15.676121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7aeac2-fbbe-4908-94ee-22ff8841e11b None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:15.687481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7aeac2-fbbe-4908-94ee-22ff8841e11b None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:15.768618 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5238f82-a579-44d4-9d68-311efc610551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:15.769298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5238f82-a579-44d4-9d68-311efc610551 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:15.769578 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5238f82-a579-44d4-9d68-311efc610551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:15.769848 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5238f82-a579-44d4-9d68-311efc610551 admin admin] PATH_INFO: `/v3/users` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:15.770511 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5238f82-a579-44d4-9d68-311efc610551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:15.772491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5238f82-a579-44d4-9d68-311efc610551 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:15.810612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1442/2858] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:51:15 2022] POST /identity/v3/users => generated 298 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:51:15.815032 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc23bd9-fa75-43c1-94b1-3010e11ba26c admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:15.825309 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc23bd9-fa75-43c1-94b1-3010e11ba26c admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:15.829853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e54325-bc8e-4fd8-9f98-c787bd16d168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:15.830507 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e54325-bc8e-4fd8-9f98-c787bd16d168 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:15.830785 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e54325-bc8e-4fd8-9f98-c787bd16d168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:15.831053 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e54325-bc8e-4fd8-9f98-c787bd16d168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:15.831682 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e54325-bc8e-4fd8-9f98-c787bd16d168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:15.833285 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e54325-bc8e-4fd8-9f98-c787bd16d168 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:15.843683 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1417/2859] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:51:15.849029 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5238f82-a579-44d4-9d68-311efc610551 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:15.866326 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5238f82-a579-44d4-9d68-311efc610551 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:15.871555 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957b184-5ec0-445f-836d-784bcd9d00bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:15.872227 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957b184-5ec0-445f-836d-784bcd9d00bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:15.872504 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957b184-5ec0-445f-836d-784bcd9d00bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:15.872779 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957b184-5ec0-445f-836d-784bcd9d00bf admin admin] PATH_INFO: `/v3/projects/76d00f0b8a86489bad1bf03513282b94/users/8b4ad7dfdb99451cad949ae9dbd88b7b/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:15.889921 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957b184-5ec0-445f-836d-784bcd9d00bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d00f0b8a86489bad1bf03513282b94, user_id=8b4ad7dfdb99451cad949ae9dbd88b7b, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:15.892874 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957b184-5ec0-445f-836d-784bcd9d00bf admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:15.909344 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1443/2860] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:51:15 2022] PUT /identity/v3/projects/76d00f0b8a86489bad1bf03513282b94/users/8b4ad7dfdb99451cad949ae9dbd88b7b/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:15.913546 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e54325-bc8e-4fd8-9f98-c787bd16d168 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:15.935774 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e54325-bc8e-4fd8-9f98-c787bd16d168 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:16.017257 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4b4af-7253-41d8-a256-2db7c8ed9975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:16.018429 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4b4af-7253-41d8-a256-2db7c8ed9975 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:16.018718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4b4af-7253-41d8-a256-2db7c8ed9975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:16.018991 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4b4af-7253-41d8-a256-2db7c8ed9975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:16.019685 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4b4af-7253-41d8-a256-2db7c8ed9975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:16.021920 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4b4af-7253-41d8-a256-2db7c8ed9975 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:16.037357 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1418/2861] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:51:16.041530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957b184-5ec0-445f-836d-784bcd9d00bf admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:16.052594 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957b184-5ec0-445f-836d-784bcd9d00bf admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:16.058047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8572c3-6301-4e7c-a1ca-2713575acb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:16.058927 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8572c3-6301-4e7c-a1ca-2713575acb78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:16.059390 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8572c3-6301-4e7c-a1ca-2713575acb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:16.059822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8572c3-6301-4e7c-a1ca-2713575acb78 admin admin] PATH_INFO: `/v3/projects/76d00f0b8a86489bad1bf03513282b94/users/8b4ad7dfdb99451cad949ae9dbd88b7b/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:16.062003 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8572c3-6301-4e7c-a1ca-2713575acb78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d00f0b8a86489bad1bf03513282b94, user_id=8b4ad7dfdb99451cad949ae9dbd88b7b, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:16.065141 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8572c3-6301-4e7c-a1ca-2713575acb78 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:16.078886 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1444/2862] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:51:16 2022] PUT /identity/v3/projects/76d00f0b8a86489bad1bf03513282b94/users/8b4ad7dfdb99451cad949ae9dbd88b7b/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:16.089676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd9253-3396-4418-ba9e-9a6ffcfc4ef6 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:16.089994 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd9253-3396-4418-ba9e-9a6ffcfc4ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:16.090274 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd9253-3396-4418-ba9e-9a6ffcfc4ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:16.122698 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-72cd9253-3396-4418-ba9e-9a6ffcfc4ef6 None None] MFA Rules not processed for user `8b4ad7dfdb99451cad949ae9dbd88b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:16.123855 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-72cd9253-3396-4418-ba9e-9a6ffcfc4ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:16.192882 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1419/2863] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:51:16.204596 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8572c3-6301-4e7c-a1ca-2713575acb78 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:16.206805 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-fc8572c3-6301-4e7c-a1ca-2713575acb78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:16.260583 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:16.266745 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8572c3-6301-4e7c-a1ca-2713575acb78 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:16.271089 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c816f7-81d0-4ff5-aeb1-e34250a3d75c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:16.271689 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c816f7-81d0-4ff5-aeb1-e34250a3d75c service cinder] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:16.271966 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c816f7-81d0-4ff5-aeb1-e34250a3d75c service cinder] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:16.272237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c816f7-81d0-4ff5-aeb1-e34250a3d75c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:16.326764 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c816f7-81d0-4ff5-aeb1-e34250a3d75c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:16.328167 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c816f7-81d0-4ff5-aeb1-e34250a3d75c service cinder] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:16.337862 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1445/2864] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:51:16.859012 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd9253-3396-4418-ba9e-9a6ffcfc4ef6 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:16.860784 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-72cd9253-3396-4418-ba9e-9a6ffcfc4ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:16.897314 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:16.939060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd9253-3396-4418-ba9e-9a6ffcfc4ef6 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:16.945687 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205ac67-de81-404a-8136-3b2de1b09c2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:16.946299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205ac67-de81-404a-8136-3b2de1b09c2a service placement] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:16.946576 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205ac67-de81-404a-8136-3b2de1b09c2a service placement] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:16.946872 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205ac67-de81-404a-8136-3b2de1b09c2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:16.948583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c205ac67-de81-404a-8136-3b2de1b09c2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:16.960838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205ac67-de81-404a-8136-3b2de1b09c2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:16.962568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205ac67-de81-404a-8136-3b2de1b09c2a service placement] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:16.973649 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1420/2865] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:51:18.397741 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c816f7-81d0-4ff5-aeb1-e34250a3d75c service cinder] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:18.399250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-63c816f7-81d0-4ff5-aeb1-e34250a3d75c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:18.462684 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c816f7-81d0-4ff5-aeb1-e34250a3d75c service cinder] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:18.466415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70478e9-4f74-4f69-9d77-f5411a38598f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:18.467028 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70478e9-4f74-4f69-9d77-f5411a38598f service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:18.467412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70478e9-4f74-4f69-9d77-f5411a38598f service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:18.467586 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70478e9-4f74-4f69-9d77-f5411a38598f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:18.469415 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-a70478e9-4f74-4f69-9d77-f5411a38598f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:18.538325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70478e9-4f74-4f69-9d77-f5411a38598f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:18.539782 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70478e9-4f74-4f69-9d77-f5411a38598f service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:18.550377 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1446/2866] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:51:20.643835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205ac67-de81-404a-8136-3b2de1b09c2a service placement] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:20.645827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-c205ac67-de81-404a-8136-3b2de1b09c2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:20.657861 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205ac67-de81-404a-8136-3b2de1b09c2a service placement] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:20.662604 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb368035-2da5-46d4-8249-480ac4879ece service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:20.663216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb368035-2da5-46d4-8249-480ac4879ece service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:20.663499 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb368035-2da5-46d4-8249-480ac4879ece service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:20.668688 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb368035-2da5-46d4-8249-480ac4879ece service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:20.680681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb368035-2da5-46d4-8249-480ac4879ece service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:20.682189 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb368035-2da5-46d4-8249-480ac4879ece service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:20.692673 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1421/2867] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:51:25.776440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5a241-88a6-464c-9ed3-6ff4f653b67a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:25.776933 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5a241-88a6-464c-9ed3-6ff4f653b67a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:25.776933 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5a241-88a6-464c-9ed3-6ff4f653b67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:25.796228 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-21c5a241-88a6-464c-9ed3-6ff4f653b67a None None] MFA Rules not processed for user `ef7a012e1303456ea7abc94e96656199`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:25.797473 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-21c5a241-88a6-464c-9ed3-6ff4f653b67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:25.826114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb368035-2da5-46d4-8249-480ac4879ece service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:25.846121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=100 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:25.856881 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1447/2868] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:51:25.894151 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb368035-2da5-46d4-8249-480ac4879ece service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:25.898819 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:25.899470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:25.899750 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:25.900023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] PATH_INFO: `/v3/users/577947ea9d964803b6db65f0f84d87f5` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:25.901167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] RBAC: Authorizing `identity:delete_user(user_id=577947ea9d964803b6db65f0f84d87f5)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:25.903144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:25.953511 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '577947ea9d964803b6db65f0f84d87f5'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:25.960782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '577947ea9d964803b6db65f0f84d87f5'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:25.961726 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '577947ea9d964803b6db65f0f84d87f5'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:25.962528 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '577947ea9d964803b6db65f0f84d87f5'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:25.971835 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '577947ea9d964803b6db65f0f84d87f5'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:25.990104 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1422/2869] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:51:25 2022] DELETE /identity/v3/users/577947ea9d964803b6db65f0f84d87f5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:51:25.994549 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b563cc-4616-47d7-b513-3da5af6af1d4 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:25.994841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b563cc-4616-47d7-b513-3da5af6af1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:25.995119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b563cc-4616-47d7-b513-3da5af6af1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:26.016368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-39b563cc-4616-47d7-b513-3da5af6af1d4 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:26.017699 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-39b563cc-4616-47d7-b513-3da5af6af1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:26.078892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1448/2870] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:51:26.088751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:26.090445 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:26.100832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b563cc-4616-47d7-b513-3da5af6af1d4 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:26.102483 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-39b563cc-4616-47d7-b513-3da5af6af1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:26.120740 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:26.156770 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7c204-9b9f-4510-9a30-07adb18bc73b admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:26.161144 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f7a18-9947-4504-8203-5ed76fffebf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:26.161762 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f7a18-9947-4504-8203-5ed76fffebf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:26.162039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f7a18-9947-4504-8203-5ed76fffebf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:26.162318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f7a18-9947-4504-8203-5ed76fffebf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:26.166483 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b563cc-4616-47d7-b513-3da5af6af1d4 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:26.171361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785868b-fb94-4bf6-8324-1d34590e9e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:26.171974 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785868b-fb94-4bf6-8324-1d34590e9e96 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:26.172106 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f7a18-9947-4504-8203-5ed76fffebf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:26.172252 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785868b-fb94-4bf6-8324-1d34590e9e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:26.172521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785868b-fb94-4bf6-8324-1d34590e9e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:26.173521 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f7a18-9947-4504-8203-5ed76fffebf5 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:26.174983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3785868b-fb94-4bf6-8324-1d34590e9e96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:26.183635 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1423/2871] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:51:26.186395 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785868b-fb94-4bf6-8324-1d34590e9e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:26.187822 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785868b-fb94-4bf6-8324-1d34590e9e96 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:26.196571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1449/2872] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:51:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 08:51:26.517057 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f7a18-9947-4504-8203-5ed76fffebf5 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:26.520676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-1c9f7a18-9947-4504-8203-5ed76fffebf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:26.536251 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f7a18-9947-4504-8203-5ed76fffebf5 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:26.541328 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:26.541960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:26.542354 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:26.542631 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] PATH_INFO: `/v3/projects/f110f673508741b3aa40aa8e7a1d33e1` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:26.543796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f110f673508741b3aa40aa8e7a1d33e1)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:26.546031 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:26.594960 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f110f673508741b3aa40aa8e7a1d33e1', 'request_id': 'req-b9c97635-2b8e-4a5f-9326-5a504086a090'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:26.602005 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f110f673508741b3aa40aa8e7a1d33e1', 'request_id': 'req-b9c97635-2b8e-4a5f-9326-5a504086a090'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:26.609613 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f110f673508741b3aa40aa8e7a1d33e1', 'request_id': 'req-b9c97635-2b8e-4a5f-9326-5a504086a090'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:26.620836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] The token cache is being invalidate because project f110f673508741b3aa40aa8e7a1d33e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:51:26.621149 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:26.621772 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:26.623229 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1424/2873] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:51:26 2022] DELETE /identity/v3/projects/f110f673508741b3aa40aa8e7a1d33e1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:26.627217 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785868b-fb94-4bf6-8324-1d34590e9e96 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:26.638072 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785868b-fb94-4bf6-8324-1d34590e9e96 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:26.693544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed10484-1cb1-425a-9cb2-567cd1adf155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:26.694178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed10484-1cb1-425a-9cb2-567cd1adf155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:26.694449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed10484-1cb1-425a-9cb2-567cd1adf155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:26.694717 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed10484-1cb1-425a-9cb2-567cd1adf155 admin admin] PATH_INFO: `/v3/users/8b4ad7dfdb99451cad949ae9dbd88b7b` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:26.695857 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed10484-1cb1-425a-9cb2-567cd1adf155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4ad7dfdb99451cad949ae9dbd88b7b)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:26.697798 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed10484-1cb1-425a-9cb2-567cd1adf155 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:26.742373 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:26.748542 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-0ed10484-1cb1-425a-9cb2-567cd1adf155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4ad7dfdb99451cad949ae9dbd88b7b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:26.761020 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-0ed10484-1cb1-425a-9cb2-567cd1adf155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4ad7dfdb99451cad949ae9dbd88b7b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:26.761790 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-0ed10484-1cb1-425a-9cb2-567cd1adf155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4ad7dfdb99451cad949ae9dbd88b7b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:26.769209 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-0ed10484-1cb1-425a-9cb2-567cd1adf155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4ad7dfdb99451cad949ae9dbd88b7b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:26.769856 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-0ed10484-1cb1-425a-9cb2-567cd1adf155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4ad7dfdb99451cad949ae9dbd88b7b'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:26.782983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1450/2874] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:51:26 2022] DELETE /identity/v3/users/8b4ad7dfdb99451cad949ae9dbd88b7b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:27.058836 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:27.060400 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:27.075684 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c97635-2b8e-4a5f-9326-5a504086a090 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:27.135186 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:27.135709 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:27.135988 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:27.136260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] PATH_INFO: `/v3/projects/76d00f0b8a86489bad1bf03513282b94` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:27.137426 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d00f0b8a86489bad1bf03513282b94)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:27.139326 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:27.188064 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d00f0b8a86489bad1bf03513282b94', 'request_id': 'req-8ba4c261-948d-4721-9351-814ed0c7e14a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:27.190501 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:27.194827 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d00f0b8a86489bad1bf03513282b94', 'request_id': 'req-8ba4c261-948d-4721-9351-814ed0c7e14a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:27.203081 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d00f0b8a86489bad1bf03513282b94', 'request_id': 'req-8ba4c261-948d-4721-9351-814ed0c7e14a'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:27.214950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] The token cache is being invalidate because project 76d00f0b8a86489bad1bf03513282b94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:51:27.215244 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:27.215881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:51:27.217396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1425/2875] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:51:27 2022] DELETE /identity/v3/projects/76d00f0b8a86489bad1bf03513282b94 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:51:36.779525 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=14,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:37.211906 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=6,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:48.084706 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3683b-dc41-498f-b109-5195353c65f7 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:48.085250 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3683b-dc41-498f-b109-5195353c65f7 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:48.085365 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3683b-dc41-498f-b109-5195353c65f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:48.104427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-7ba3683b-dc41-498f-b109-5195353c65f7 None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:51:48.105668 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7ba3683b-dc41-498f-b109-5195353c65f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:48.163568 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1451/2876] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:51:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:51:48.210517 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:51:48.211986 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:51:48.282563 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4c261-948d-4721-9351-814ed0c7e14a admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:51:48.288060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09cd8f7-cd44-4d9b-88bf-7905c59aceee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:51:48.288666 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09cd8f7-cd44-4d9b-88bf-7905c59aceee service swift] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:51:48.288945 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09cd8f7-cd44-4d9b-88bf-7905c59aceee service swift] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:51:48.289243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09cd8f7-cd44-4d9b-88bf-7905c59aceee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:51:48.299318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09cd8f7-cd44-4d9b-88bf-7905c59aceee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:51:48.300931 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09cd8f7-cd44-4d9b-88bf-7905c59aceee service swift] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:51:48.310281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1426/2877] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:51:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:51:58.155311 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=25 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:51:58.306780 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=32 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:52:03.660845 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fae573-d8b9-4527-8e09-8267c06b76eb None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:03.661341 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fae573-d8b9-4527-8e09-8267c06b76eb None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:03.661426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fae573-d8b9-4527-8e09-8267c06b76eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:03.681051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-62fae573-d8b9-4527-8e09-8267c06b76eb None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:52:03.682298 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-62fae573-d8b9-4527-8e09-8267c06b76eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:03.689819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1452/2878] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 08:52:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:52:03.701978 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09cd8f7-cd44-4d9b-88bf-7905c59aceee service swift] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:03.713135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09cd8f7-cd44-4d9b-88bf-7905c59aceee service swift] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:03.718167 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a781cee-c450-4b33-906c-bb7d3a13423f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:03.718759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a781cee-c450-4b33-906c-bb7d3a13423f service swift] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:03.719039 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a781cee-c450-4b33-906c-bb7d3a13423f service swift] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:03.719316 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a781cee-c450-4b33-906c-bb7d3a13423f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:03.728560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a781cee-c450-4b33-906c-bb7d3a13423f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:03.730175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a781cee-c450-4b33-906c-bb7d3a13423f service swift] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:03.738241 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1427/2879] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:52:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:52:05.625351 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fae573-d8b9-4527-8e09-8267c06b76eb None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:05.626873 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-62fae573-d8b9-4527-8e09-8267c06b76eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:05.677508 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a781cee-c450-4b33-906c-bb7d3a13423f service swift] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:05.679932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9a781cee-c450-4b33-906c-bb7d3a13423f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:05.707693 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fae573-d8b9-4527-8e09-8267c06b76eb None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:05.713339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf6075-724e-4381-8eed-12b685a66225 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:05.713774 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf6075-724e-4381-8eed-12b685a66225 service placement] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:05.714127 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf6075-724e-4381-8eed-12b685a66225 service placement] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:05.714406 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf6075-724e-4381-8eed-12b685a66225 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:05.716163 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-dcaf6075-724e-4381-8eed-12b685a66225 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:05.728629 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf6075-724e-4381-8eed-12b685a66225 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:05.730243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf6075-724e-4381-8eed-12b685a66225 service placement] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:05.740871 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1453/2880] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:52:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:52:05.752249 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a781cee-c450-4b33-906c-bb7d3a13423f service swift] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:05.757529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd63052-babc-46a0-9be8-4bf97088cfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:05.758173 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd63052-babc-46a0-9be8-4bf97088cfcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:05.758455 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd63052-babc-46a0-9be8-4bf97088cfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:05.758782 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd63052-babc-46a0-9be8-4bf97088cfcc admin admin] PATH_INFO: `/v3/users/ef7a012e1303456ea7abc94e96656199` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:05.760135 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd63052-babc-46a0-9be8-4bf97088cfcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef7a012e1303456ea7abc94e96656199)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:05.762416 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd63052-babc-46a0-9be8-4bf97088cfcc admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:05.810949 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7bd63052-babc-46a0-9be8-4bf97088cfcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef7a012e1303456ea7abc94e96656199'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:05.817941 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7bd63052-babc-46a0-9be8-4bf97088cfcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef7a012e1303456ea7abc94e96656199'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:05.818614 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7bd63052-babc-46a0-9be8-4bf97088cfcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef7a012e1303456ea7abc94e96656199'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:05.819215 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7bd63052-babc-46a0-9be8-4bf97088cfcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef7a012e1303456ea7abc94e96656199'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:05.827689 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-7bd63052-babc-46a0-9be8-4bf97088cfcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef7a012e1303456ea7abc94e96656199'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:05.845830 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1428/2881] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:52:05 2022] DELETE /identity/v3/users/ef7a012e1303456ea7abc94e96656199 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:52:06.013776 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf6075-724e-4381-8eed-12b685a66225 service placement] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:06.025061 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf6075-724e-4381-8eed-12b685a66225 service placement] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:06.086175 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df90b09-e9b0-4a57-888c-6462a3a0383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:06.086807 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df90b09-e9b0-4a57-888c-6462a3a0383f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:06.087090 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df90b09-e9b0-4a57-888c-6462a3a0383f admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:06.087366 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df90b09-e9b0-4a57-888c-6462a3a0383f admin admin] PATH_INFO: `/v3/projects/63011f747824422da9f24e5aa43a57ad` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:06.088492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df90b09-e9b0-4a57-888c-6462a3a0383f admin admin] RBAC: Authorizing `identity:delete_project(project_id=63011f747824422da9f24e5aa43a57ad)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:06.090420 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df90b09-e9b0-4a57-888c-6462a3a0383f admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:06.138421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9df90b09-e9b0-4a57-888c-6462a3a0383f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63011f747824422da9f24e5aa43a57ad', 'request_id': 'req-9df90b09-e9b0-4a57-888c-6462a3a0383f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:06.144080 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9df90b09-e9b0-4a57-888c-6462a3a0383f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63011f747824422da9f24e5aa43a57ad', 'request_id': 'req-9df90b09-e9b0-4a57-888c-6462a3a0383f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:06.151261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9df90b09-e9b0-4a57-888c-6462a3a0383f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63011f747824422da9f24e5aa43a57ad', 'request_id': 'req-9df90b09-e9b0-4a57-888c-6462a3a0383f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:06.163345 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9df90b09-e9b0-4a57-888c-6462a3a0383f admin admin] The token cache is being invalidate because project 63011f747824422da9f24e5aa43a57ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:52:06.163622 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9df90b09-e9b0-4a57-888c-6462a3a0383f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:06.164248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-9df90b09-e9b0-4a57-888c-6462a3a0383f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:06.165761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1454/2882] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:52:06 2022] DELETE /identity/v3/projects/63011f747824422da9f24e5aa43a57ad => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:52:06.171015 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38952671-2e90-4201-b01d-9df9b0cc3e88 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:06.171327 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38952671-2e90-4201-b01d-9df9b0cc3e88 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:06.171605 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38952671-2e90-4201-b01d-9df9b0cc3e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:06.191318 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-38952671-2e90-4201-b01d-9df9b0cc3e88 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:52:06.192930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-38952671-2e90-4201-b01d-9df9b0cc3e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:06.251369 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1429/2883] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:52:06.260825 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a161ea1-97e4-4305-bd01-294be858201e None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:06.261140 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a161ea1-97e4-4305-bd01-294be858201e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:06.261426 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a161ea1-97e4-4305-bd01-294be858201e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:06.281766 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-3a161ea1-97e4-4305-bd01-294be858201e None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:52:06.282988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-3a161ea1-97e4-4305-bd01-294be858201e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:06.290977 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1455/2884] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:52:06.295436 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38952671-2e90-4201-b01d-9df9b0cc3e88 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:06.305184 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38952671-2e90-4201-b01d-9df9b0cc3e88 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:06.309672 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d0e40-ec3e-4610-9a09-5cd8a227b928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:06.310259 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d0e40-ec3e-4610-9a09-5cd8a227b928 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:06.310537 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d0e40-ec3e-4610-9a09-5cd8a227b928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:06.310807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d0e40-ec3e-4610-9a09-5cd8a227b928 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:06.311448 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d0e40-ec3e-4610-9a09-5cd8a227b928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:52:06.311747 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d0e40-ec3e-4610-9a09-5cd8a227b928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:06.313344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d0e40-ec3e-4610-9a09-5cd8a227b928 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:06.314810 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:52:06.326961 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1430/2885] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:52:06.331108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a161ea1-97e4-4305-bd01-294be858201e None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:06.341150 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a161ea1-97e4-4305-bd01-294be858201e None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:06.345833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c080ea4-bd32-48f9-af54-29ef979e20de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:06.346425 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c080ea4-bd32-48f9-af54-29ef979e20de admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:06.346702 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c080ea4-bd32-48f9-af54-29ef979e20de admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:06.346971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c080ea4-bd32-48f9-af54-29ef979e20de admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:06.347603 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c080ea4-bd32-48f9-af54-29ef979e20de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:06.349505 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c080ea4-bd32-48f9-af54-29ef979e20de admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:06.353896 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:52:06.380323 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1456/2886] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:52:06 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:52:06.384523 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d0e40-ec3e-4610-9a09-5cd8a227b928 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:06.394409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d0e40-ec3e-4610-9a09-5cd8a227b928 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:06.451881 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591e8e0-9369-4106-b37d-c1ad2a1ceaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:06.452415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591e8e0-9369-4106-b37d-c1ad2a1ceaf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:06.452696 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591e8e0-9369-4106-b37d-c1ad2a1ceaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:06.452970 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591e8e0-9369-4106-b37d-c1ad2a1ceaf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:06.453637 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591e8e0-9369-4106-b37d-c1ad2a1ceaf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:06.455585 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591e8e0-9369-4106-b37d-c1ad2a1ceaf9 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:06.477853 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1431/2887] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:52:06 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:52:06.481786 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c080ea4-bd32-48f9-af54-29ef979e20de admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:06.492855 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c080ea4-bd32-48f9-af54-29ef979e20de admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:06.497530 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc2c3-2645-4724-b23f-16dada994e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:06.498128 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc2c3-2645-4724-b23f-16dada994e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:06.498407 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc2c3-2645-4724-b23f-16dada994e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:06.498678 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc2c3-2645-4724-b23f-16dada994e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:06.499314 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc2c3-2645-4724-b23f-16dada994e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:06.500893 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc2c3-2645-4724-b23f-16dada994e0a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:06.511609 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1457/2888] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:52:06.515824 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591e8e0-9369-4106-b37d-c1ad2a1ceaf9 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:06.525722 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591e8e0-9369-4106-b37d-c1ad2a1ceaf9 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:06.530568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35267ced-2d9e-4f43-b9cb-d0e46a9784ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:06.531175 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267ced-2d9e-4f43-b9cb-d0e46a9784ba admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:06.531453 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267ced-2d9e-4f43-b9cb-d0e46a9784ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:06.531724 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267ced-2d9e-4f43-b9cb-d0e46a9784ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:06.532368 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35267ced-2d9e-4f43-b9cb-d0e46a9784ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:06.533982 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35267ced-2d9e-4f43-b9cb-d0e46a9784ba admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:06.545245 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1432/2889] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:52:06.549483 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc2c3-2645-4724-b23f-16dada994e0a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:06.560300 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc2c3-2645-4724-b23f-16dada994e0a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:06.565139 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4035a-15c4-433e-80f5-f00c8aed45d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:06.565907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4035a-15c4-433e-80f5-f00c8aed45d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:06.566194 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4035a-15c4-433e-80f5-f00c8aed45d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:06.566497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4035a-15c4-433e-80f5-f00c8aed45d0 admin admin] PATH_INFO: `/v3/projects/b048ee37e3f044c989511c64fb66f9be/users/27f124075a3047d39cfee575f9724ec0/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:06.585224 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e4035a-15c4-433e-80f5-f00c8aed45d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b048ee37e3f044c989511c64fb66f9be, user_id=27f124075a3047d39cfee575f9724ec0, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:06.590154 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e4035a-15c4-433e-80f5-f00c8aed45d0 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:06.605506 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1458/2890] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:52:06 2022] PUT /identity/v3/projects/b048ee37e3f044c989511c64fb66f9be/users/27f124075a3047d39cfee575f9724ec0/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:52:06.617339 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2487109-87e7-49d0-9188-cb209050a3c6 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:06.617627 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2487109-87e7-49d0-9188-cb209050a3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:06.617935 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2487109-87e7-49d0-9188-cb209050a3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:06.646471 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-b2487109-87e7-49d0-9188-cb209050a3c6 None None] MFA Rules not processed for user `27f124075a3047d39cfee575f9724ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:52:06.647681 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b2487109-87e7-49d0-9188-cb209050a3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:06.714097 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1433/2891] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:52:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:52:06.725325 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4035a-15c4-433e-80f5-f00c8aed45d0 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:06.727118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-62e4035a-15c4-433e-80f5-f00c8aed45d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:06.806267 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4035a-15c4-433e-80f5-f00c8aed45d0 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:06.809968 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaffd93-b468-472b-8317-3d2ec58ae00d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff22c2632c614fca8008feab93c8fa2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:06.810571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaffd93-b468-472b-8317-3d2ec58ae00d service glance] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:06.810850 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaffd93-b468-472b-8317-3d2ec58ae00d service glance] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:06.811121 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaffd93-b468-472b-8317-3d2ec58ae00d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:06.820603 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaffd93-b468-472b-8317-3d2ec58ae00d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:06.822081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaffd93-b468-472b-8317-3d2ec58ae00d service glance] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:06.830956 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1459/2892] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:52:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:52:16.710625 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:52:16.827075 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:52:49.739676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2487109-87e7-49d0-9188-cb209050a3c6 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:49.807004 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2487109-87e7-49d0-9188-cb209050a3c6 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:49.811525 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:49.812160 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:49.812439 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:49.812714 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] PATH_INFO: `/v3/users/27f124075a3047d39cfee575f9724ec0` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:49.813869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] RBAC: Authorizing `identity:delete_user(user_id=27f124075a3047d39cfee575f9724ec0)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:49.815781 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:49.865321 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27f124075a3047d39cfee575f9724ec0'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:49.873022 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27f124075a3047d39cfee575f9724ec0'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:49.873693 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27f124075a3047d39cfee575f9724ec0'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:49.874303 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27f124075a3047d39cfee575f9724ec0'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:49.881059 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27f124075a3047d39cfee575f9724ec0'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:49.899252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1434/2893] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:52:49 2022] DELETE /identity/v3/users/27f124075a3047d39cfee575f9724ec0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:52:49.903749 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21c618-89dd-475a-aafd-ed3cc5319d0a None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:49.904380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21c618-89dd-475a-aafd-ed3cc5319d0a None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:49.904380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21c618-89dd-475a-aafd-ed3cc5319d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:49.923318 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-2a21c618-89dd-475a-aafd-ed3cc5319d0a None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:52:49.924524 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2a21c618-89dd-475a-aafd-ed3cc5319d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:49.984205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1460/2894] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:52:49.994389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:49.996233 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:50.055085 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779fbaf-3893-435b-90a2-6f356126e54d admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:50.058885 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cba65-a1ba-4cb0-a317-d133eb86b33d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:50.059432 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cba65-a1ba-4cb0-a317-d133eb86b33d service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:50.059716 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cba65-a1ba-4cb0-a317-d133eb86b33d service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:50.059985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cba65-a1ba-4cb0-a317-d133eb86b33d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:50.069164 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8cba65-a1ba-4cb0-a317-d133eb86b33d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:50.070590 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8cba65-a1ba-4cb0-a317-d133eb86b33d service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:50.080007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1435/2895] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:52:50.351665 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21c618-89dd-475a-aafd-ed3cc5319d0a None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:50.353104 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-2a21c618-89dd-475a-aafd-ed3cc5319d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:50.365978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21c618-89dd-475a-aafd-ed3cc5319d0a None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:50.373246 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03069c9d-22ce-4220-a474-dc76cc21207c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:50.373936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03069c9d-22ce-4220-a474-dc76cc21207c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:50.374383 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03069c9d-22ce-4220-a474-dc76cc21207c admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:50.374661 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03069c9d-22ce-4220-a474-dc76cc21207c admin admin] PATH_INFO: `/v3/projects/b048ee37e3f044c989511c64fb66f9be` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:50.375844 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03069c9d-22ce-4220-a474-dc76cc21207c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b048ee37e3f044c989511c64fb66f9be)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:50.377978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03069c9d-22ce-4220-a474-dc76cc21207c admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:50.425467 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-03069c9d-22ce-4220-a474-dc76cc21207c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b048ee37e3f044c989511c64fb66f9be', 'request_id': 'req-03069c9d-22ce-4220-a474-dc76cc21207c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:50.431534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-03069c9d-22ce-4220-a474-dc76cc21207c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b048ee37e3f044c989511c64fb66f9be', 'request_id': 'req-03069c9d-22ce-4220-a474-dc76cc21207c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:50.437753 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-03069c9d-22ce-4220-a474-dc76cc21207c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b048ee37e3f044c989511c64fb66f9be', 'request_id': 'req-03069c9d-22ce-4220-a474-dc76cc21207c'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:50.450783 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-03069c9d-22ce-4220-a474-dc76cc21207c admin admin] The token cache is being invalidate because project b048ee37e3f044c989511c64fb66f9be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:52:50.451074 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-03069c9d-22ce-4220-a474-dc76cc21207c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:50.451708 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-03069c9d-22ce-4220-a474-dc76cc21207c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:52:50.453222 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1461/2896] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:52:50 2022] DELETE /identity/v3/projects/b048ee37e3f044c989511c64fb66f9be => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:52:50.458473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023599d-8a90-4942-93f1-1c522f3e64f9 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:50.458755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023599d-8a90-4942-93f1-1c522f3e64f9 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:50.459023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023599d-8a90-4942-93f1-1c522f3e64f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:50.478069 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-9023599d-8a90-4942-93f1-1c522f3e64f9 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:52:50.479243 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-9023599d-8a90-4942-93f1-1c522f3e64f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:50.496595 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:52:50.541676 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1436/2897] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:52:50.549644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db525a-f5e9-4c41-b840-0fb27c8a6ba8 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:50.549936 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db525a-f5e9-4c41-b840-0fb27c8a6ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:50.550237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db525a-f5e9-4c41-b840-0fb27c8a6ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:50.571186 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-63db525a-f5e9-4c41-b840-0fb27c8a6ba8 None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:52:50.572413 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-63db525a-f5e9-4c41-b840-0fb27c8a6ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:50.580491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1462/2898] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:52:50.584874 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023599d-8a90-4942-93f1-1c522f3e64f9 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:50.594732 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023599d-8a90-4942-93f1-1c522f3e64f9 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:50.599360 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc4f8b-ec3a-450e-abde-74b46aed25b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:50.599973 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc4f8b-ec3a-450e-abde-74b46aed25b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:50.600286 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc4f8b-ec3a-450e-abde-74b46aed25b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:50.600561 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc4f8b-ec3a-450e-abde-74b46aed25b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:50.601254 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc4f8b-ec3a-450e-abde-74b46aed25b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81483) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 08:52:50.601560 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc4f8b-ec3a-450e-abde-74b46aed25b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:50.603158 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc4f8b-ec3a-450e-abde-74b46aed25b8 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:50.615038 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1437/2899] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 08:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 08:52:50.619934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db525a-f5e9-4c41-b840-0fb27c8a6ba8 None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:50.629081 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db525a-f5e9-4c41-b840-0fb27c8a6ba8 None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:50.634364 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03f920-a269-44e9-80d9-d07d4476484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:50.634954 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03f920-a269-44e9-80d9-d07d4476484a admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:50.635231 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03f920-a269-44e9-80d9-d07d4476484a admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:50.635499 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03f920-a269-44e9-80d9-d07d4476484a admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:50.636123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03f920-a269-44e9-80d9-d07d4476484a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:50.638033 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03f920-a269-44e9-80d9-d07d4476484a admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:50.664739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1463/2900] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:52:50 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:52:50.669430 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc4f8b-ec3a-450e-abde-74b46aed25b8 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:50.679942 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc4f8b-ec3a-450e-abde-74b46aed25b8 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:50.736578 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8268eaa-b57b-4329-9895-62bd4de9a121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:50.737212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8268eaa-b57b-4329-9895-62bd4de9a121 admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:50.737498 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8268eaa-b57b-4329-9895-62bd4de9a121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:50.737773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8268eaa-b57b-4329-9895-62bd4de9a121 admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:50.738397 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8268eaa-b57b-4329-9895-62bd4de9a121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:50.740307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8268eaa-b57b-4329-9895-62bd4de9a121 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:50.762470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1438/2901] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:52:50 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:52:50.767041 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03f920-a269-44e9-80d9-d07d4476484a admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:50.777972 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03f920-a269-44e9-80d9-d07d4476484a admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:50.782662 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06a63d-668b-47bb-a8e6-1a321f3a04ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:50.783261 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06a63d-668b-47bb-a8e6-1a321f3a04ce admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:50.783540 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06a63d-668b-47bb-a8e6-1a321f3a04ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:50.783865 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06a63d-668b-47bb-a8e6-1a321f3a04ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:50.784586 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06a63d-668b-47bb-a8e6-1a321f3a04ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:50.786119 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06a63d-668b-47bb-a8e6-1a321f3a04ce admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:50.797243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1464/2902] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:52:50.801474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8268eaa-b57b-4329-9895-62bd4de9a121 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:50.811927 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8268eaa-b57b-4329-9895-62bd4de9a121 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:50.816457 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68a5fb-6e77-4b4e-9408-27be79b51e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:50.817048 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68a5fb-6e77-4b4e-9408-27be79b51e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:50.817359 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68a5fb-6e77-4b4e-9408-27be79b51e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:50.817632 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68a5fb-6e77-4b4e-9408-27be79b51e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:50.818263 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68a5fb-6e77-4b4e-9408-27be79b51e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:50.819847 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68a5fb-6e77-4b4e-9408-27be79b51e4f admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:50.830721 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1439/2903] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:52:50.834837 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06a63d-668b-47bb-a8e6-1a321f3a04ce admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:50.844730 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06a63d-668b-47bb-a8e6-1a321f3a04ce admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:50.849408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17873532-eb5e-4144-99fa-ca31275f2d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:50.849984 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17873532-eb5e-4144-99fa-ca31275f2d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:50.850746 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17873532-eb5e-4144-99fa-ca31275f2d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:50.851022 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17873532-eb5e-4144-99fa-ca31275f2d0e admin admin] PATH_INFO: `/v3/projects/a5865848bafe480db52b92ab3323645f/users/7a41fc47ed5447af88710dd6e74523ce/roles/4c80373ee4c34171a7f29e5ab6c09ce0` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:50.865846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17873532-eb5e-4144-99fa-ca31275f2d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5865848bafe480db52b92ab3323645f, user_id=7a41fc47ed5447af88710dd6e74523ce, role_id=4c80373ee4c34171a7f29e5ab6c09ce0)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:50.869169 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17873532-eb5e-4144-99fa-ca31275f2d0e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:50.885301 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1465/2904] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:52:50 2022] PUT /identity/v3/projects/a5865848bafe480db52b92ab3323645f/users/7a41fc47ed5447af88710dd6e74523ce/roles/4c80373ee4c34171a7f29e5ab6c09ce0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:52:50.890475 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa2b6b-4954-4c2e-be6b-b35d71af76ae None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:50.890761 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa2b6b-4954-4c2e-be6b-b35d71af76ae None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:50.891035 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa2b6b-4954-4c2e-be6b-b35d71af76ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:50.909760 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-01fa2b6b-4954-4c2e-be6b-b35d71af76ae None None] MFA Rules not processed for user `3300b0df5e5741c8b0d940d0202bb604`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:52:50.910964 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-01fa2b6b-4954-4c2e-be6b-b35d71af76ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:50.969772 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1440/2905] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:52:50.980486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17873532-eb5e-4144-99fa-ca31275f2d0e admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:50.982536 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-17873532-eb5e-4144-99fa-ca31275f2d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:51.005303 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:52:51.041840 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17873532-eb5e-4144-99fa-ca31275f2d0e admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:51.045795 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a56ff-75b6-4f9e-98ba-35378b81485e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:51.046402 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a56ff-75b6-4f9e-98ba-35378b81485e service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:51.046688 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a56ff-75b6-4f9e-98ba-35378b81485e service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:51.046959 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a56ff-75b6-4f9e-98ba-35378b81485e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:51.056532 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a56ff-75b6-4f9e-98ba-35378b81485e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:51.057978 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a56ff-75b6-4f9e-98ba-35378b81485e service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:51.067319 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1466/2906] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:52:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:52:55.079182 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b2d19-c279-402a-a53b-0fcf101c6c88 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:55.079706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b2d19-c279-402a-a53b-0fcf101c6c88 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:55.079706 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b2d19-c279-402a-a53b-0fcf101c6c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:55.109625 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-927b2d19-c279-402a-a53b-0fcf101c6c88 None None] MFA Rules not processed for user `7a41fc47ed5447af88710dd6e74523ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:52:55.110890 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-927b2d19-c279-402a-a53b-0fcf101c6c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:52:55.129114 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:52:55.187350 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1441/2907] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:52:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:52:55.192284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a56ff-75b6-4f9e-98ba-35378b81485e service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:55.202971 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a56ff-75b6-4f9e-98ba-35378b81485e service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:55.207948 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac888f-d024-40d5-bc99-d78bda99c9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:55.208654 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac888f-d024-40d5-bc99-d78bda99c9d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:55.209107 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac888f-d024-40d5-bc99-d78bda99c9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:55.209492 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac888f-d024-40d5-bc99-d78bda99c9d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:55.210284 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac888f-d024-40d5-bc99-d78bda99c9d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:55.212401 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac888f-d024-40d5-bc99-d78bda99c9d8 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:55.246108 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1467/2908] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 08:52:55 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 08:52:55.247440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b2d19-c279-402a-a53b-0fcf101c6c88 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:55.258833 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b2d19-c279-402a-a53b-0fcf101c6c88 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:55.321985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc4ed9-7658-460c-8227-f5fcca5fe1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:55.322751 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc4ed9-7658-460c-8227-f5fcca5fe1ea admin admin] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:55.323103 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc4ed9-7658-460c-8227-f5fcca5fe1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:55.323449 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc4ed9-7658-460c-8227-f5fcca5fe1ea admin admin] PATH_INFO: `/v3/users` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:55.324236 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc4ed9-7658-460c-8227-f5fcca5fe1ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:55.326317 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc4ed9-7658-460c-8227-f5fcca5fe1ea admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:55.348923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1442/2909] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 08:52:55 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 08:52:55.353666 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac888f-d024-40d5-bc99-d78bda99c9d8 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:55.364339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac888f-d024-40d5-bc99-d78bda99c9d8 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:55.369617 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929a68d-a2c4-439b-bd0e-cbc0e62e01c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:55.370243 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929a68d-a2c4-439b-bd0e-cbc0e62e01c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:55.370516 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929a68d-a2c4-439b-bd0e-cbc0e62e01c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:55.370784 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929a68d-a2c4-439b-bd0e-cbc0e62e01c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:55.371421 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929a68d-a2c4-439b-bd0e-cbc0e62e01c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:55.372994 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929a68d-a2c4-439b-bd0e-cbc0e62e01c9 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:55.385051 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1468/2910] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:52:55.390479 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc4ed9-7658-460c-8227-f5fcca5fe1ea admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:55.401473 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc4ed9-7658-460c-8227-f5fcca5fe1ea admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:55.406120 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3361fc6-86ff-4962-bedd-6543465ed2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:55.406753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3361fc6-86ff-4962-bedd-6543465ed2c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:55.407029 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3361fc6-86ff-4962-bedd-6543465ed2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:55.407298 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3361fc6-86ff-4962-bedd-6543465ed2c0 admin admin] PATH_INFO: `/v3/projects/f445710aafeb4a859f155a27d0afb60a/users/737518ef022f44c59dd527129048e1ae/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:55.422248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3361fc6-86ff-4962-bedd-6543465ed2c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f445710aafeb4a859f155a27d0afb60a, user_id=737518ef022f44c59dd527129048e1ae, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:55.424894 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3361fc6-86ff-4962-bedd-6543465ed2c0 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:55.437869 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1443/2911] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:52:55 2022] PUT /identity/v3/projects/f445710aafeb4a859f155a27d0afb60a/users/737518ef022f44c59dd527129048e1ae/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:52:55.441833 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929a68d-a2c4-439b-bd0e-cbc0e62e01c9 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:55.452281 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929a68d-a2c4-439b-bd0e-cbc0e62e01c9 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:55.507983 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e1606-7e1b-46e6-ab55-c17408b755ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:55.508607 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e1606-7e1b-46e6-ab55-c17408b755ae admin admin] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:55.508884 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e1606-7e1b-46e6-ab55-c17408b755ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:55.509191 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e1606-7e1b-46e6-ab55-c17408b755ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:55.509827 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e1606-7e1b-46e6-ab55-c17408b755ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:55.511414 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e1606-7e1b-46e6-ab55-c17408b755ae admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:55.524264 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1469/2912] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 08:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 08:52:55.528228 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3361fc6-86ff-4962-bedd-6543465ed2c0 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:52:55.538808 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3361fc6-86ff-4962-bedd-6543465ed2c0 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:52:55.543718 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efbc92-46ec-45e0-ac34-0063f24f4373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:52:55.544366 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efbc92-46ec-45e0-ac34-0063f24f4373 admin admin] REQUEST_METHOD: `PUT` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:52:55.544642 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efbc92-46ec-45e0-ac34-0063f24f4373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:52:55.544922 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efbc92-46ec-45e0-ac34-0063f24f4373 admin admin] PATH_INFO: `/v3/projects/f445710aafeb4a859f155a27d0afb60a/users/737518ef022f44c59dd527129048e1ae/roles/9c7a8295343a410f8b835c8f65408be6` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:52:55.546888 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efbc92-46ec-45e0-ac34-0063f24f4373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f445710aafeb4a859f155a27d0afb60a, user_id=737518ef022f44c59dd527129048e1ae, role_id=9c7a8295343a410f8b835c8f65408be6)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:52:55.549568 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efbc92-46ec-45e0-ac34-0063f24f4373 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:52:55.561060 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1444/2913] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 08:52:55 2022] PUT /identity/v3/projects/f445710aafeb4a859f155a27d0afb60a/users/737518ef022f44c59dd527129048e1ae/roles/9c7a8295343a410f8b835c8f65408be6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:53:00.195672 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41271-0a6a-4a1e-8297-8cafcba96076 None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:00.197534 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41271-0a6a-4a1e-8297-8cafcba96076 None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:00.197895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41271-0a6a-4a1e-8297-8cafcba96076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:00.246519 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-86e41271-0a6a-4a1e-8297-8cafcba96076 None None] MFA Rules not processed for user `737518ef022f44c59dd527129048e1ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:53:00.248157 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-86e41271-0a6a-4a1e-8297-8cafcba96076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:00.340928 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1470/2914] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:53:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 08:53:00.353529 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efbc92-46ec-45e0-ac34-0063f24f4373 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:53:00.355520 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-15efbc92-46ec-45e0-ac34-0063f24f4373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:00.418968 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efbc92-46ec-45e0-ac34-0063f24f4373 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:53:00.422825 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b5a9b-ea44-471f-be8d-1816a6ff67ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff22c2632c614fca8008feab93c8fa2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:53:00.423447 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b5a9b-ea44-471f-be8d-1816a6ff67ed service glance] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:00.423758 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b5a9b-ea44-471f-be8d-1816a6ff67ed service glance] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:00.424056 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b5a9b-ea44-471f-be8d-1816a6ff67ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:00.455255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:53:00.494100 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b5a9b-ea44-471f-be8d-1816a6ff67ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:53:00.495573 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b5a9b-ea44-471f-be8d-1816a6ff67ed service glance] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:53:00.505344 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1445/2915] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:53:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:53:00.641123 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6bb9e-9479-4b39-8777-31fdbc61da2c None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:00.641554 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6bb9e-9479-4b39-8777-31fdbc61da2c None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:00.641916 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6bb9e-9479-4b39-8777-31fdbc61da2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:00.662236 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5ea6bb9e-9479-4b39-8777-31fdbc61da2c None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:53:00.663648 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5ea6bb9e-9479-4b39-8777-31fdbc61da2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:00.670991 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:53:00.729271 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1471/2916] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:53:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:53:00.751887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b5a9b-ea44-471f-be8d-1816a6ff67ed service glance] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:53:00.753466 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-fb6b5a9b-ea44-471f-be8d-1816a6ff67ed service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:00.834759 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b5a9b-ea44-471f-be8d-1816a6ff67ed service glance] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:53:00.839659 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ead38a-107c-453c-a5f9-b671bb30e996 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:53:00.840252 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ead38a-107c-453c-a5f9-b671bb30e996 service swift] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:00.840527 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ead38a-107c-453c-a5f9-b671bb30e996 service swift] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:00.840796 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ead38a-107c-453c-a5f9-b671bb30e996 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:00.848832 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ead38a-107c-453c-a5f9-b671bb30e996 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:53:00.850459 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ead38a-107c-453c-a5f9-b671bb30e996 service swift] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:53:00.858407 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1446/2917] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:53:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:53:09.821370 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2caa25-2ae6-4f4a-beee-ecdf5889e9ea None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:09.821963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2caa25-2ae6-4f4a-beee-ecdf5889e9ea None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:09.821963 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2caa25-2ae6-4f4a-beee-ecdf5889e9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:09.841440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-5f2caa25-2ae6-4f4a-beee-ecdf5889e9ea None None] MFA Rules not processed for user `7a41fc47ed5447af88710dd6e74523ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:53:09.842644 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5f2caa25-2ae6-4f4a-beee-ecdf5889e9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:09.850111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1472/2918] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:53:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:53:09.890137 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ead38a-107c-453c-a5f9-b671bb30e996 service swift] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:53:09.900389 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ead38a-107c-453c-a5f9-b671bb30e996 service swift] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:53:09.904113 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0118d-ee6b-4cfd-812f-93f5c30ba0d4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff22c2632c614fca8008feab93c8fa2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:53:09.904710 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0118d-ee6b-4cfd-812f-93f5c30ba0d4 service glance] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:09.904990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0118d-ee6b-4cfd-812f-93f5c30ba0d4 service glance] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:09.905332 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0118d-ee6b-4cfd-812f-93f5c30ba0d4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:09.907223 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a6f0118d-ee6b-4cfd-812f-93f5c30ba0d4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:09.984227 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0118d-ee6b-4cfd-812f-93f5c30ba0d4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:53:09.985738 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0118d-ee6b-4cfd-812f-93f5c30ba0d4 service glance] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:53:09.996898 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1447/2919] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:53:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:53:10.540440 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2caa25-2ae6-4f4a-beee-ecdf5889e9ea None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:53:10.542286 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-5f2caa25-2ae6-4f4a-beee-ecdf5889e9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:10.609579 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2caa25-2ae6-4f4a-beee-ecdf5889e9ea None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:53:10.613940 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378eac6d-86d1-4c31-b574-b0c3a5b90e28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:53:10.615027 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378eac6d-86d1-4c31-b574-b0c3a5b90e28 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:10.615273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378eac6d-86d1-4c31-b574-b0c3a5b90e28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:10.615625 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378eac6d-86d1-4c31-b574-b0c3a5b90e28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:10.617447 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-378eac6d-86d1-4c31-b574-b0c3a5b90e28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:10.632016 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378eac6d-86d1-4c31-b574-b0c3a5b90e28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:53:10.633491 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378eac6d-86d1-4c31-b574-b0c3a5b90e28 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:53:10.645818 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1473/2920] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:53:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:53:11.974334 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efae77d-b0bb-4f19-8753-bf586e157078 None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:11.974838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efae77d-b0bb-4f19-8753-bf586e157078 None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:11.975007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efae77d-b0bb-4f19-8753-bf586e157078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:11.996621 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-5efae77d-b0bb-4f19-8753-bf586e157078 None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:53:11.997902 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-5efae77d-b0bb-4f19-8753-bf586e157078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:12.006650 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1448/2921] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 08:53:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:53:12.020480 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378eac6d-86d1-4c31-b574-b0c3a5b90e28 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:53:12.038177 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378eac6d-86d1-4c31-b574-b0c3a5b90e28 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:53:12.044526 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e85bf7-0c89-4c0e-a6e4-df2e79364c65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:53:12.045178 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e85bf7-0c89-4c0e-a6e4-df2e79364c65 service swift] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:12.045463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e85bf7-0c89-4c0e-a6e4-df2e79364c65 service swift] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:12.045935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e85bf7-0c89-4c0e-a6e4-df2e79364c65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:12.055380 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e85bf7-0c89-4c0e-a6e4-df2e79364c65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:53:12.056957 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e85bf7-0c89-4c0e-a6e4-df2e79364c65 service swift] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:53:12.065626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1474/2922] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:53:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:53:21.989281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=85 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:53:22.062330 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=55 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:53:42.394070 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efae77d-b0bb-4f19-8753-bf586e157078 None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:53:42.404165 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efae77d-b0bb-4f19-8753-bf586e157078 None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:53:42.408101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0882ba-e630-40f0-b984-4b1a12fd5b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:53:42.408708 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0882ba-e630-40f0-b984-4b1a12fd5b2b service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:42.408987 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0882ba-e630-40f0-b984-4b1a12fd5b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:42.409292 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0882ba-e630-40f0-b984-4b1a12fd5b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:42.419838 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0882ba-e630-40f0-b984-4b1a12fd5b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:53:42.421306 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0882ba-e630-40f0-b984-4b1a12fd5b2b service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:53:42.432596 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1449/2923] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:53:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:53:45.996716 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e85bf7-0c89-4c0e-a6e4-df2e79364c65 service swift] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:53:45.998262 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-95e85bf7-0c89-4c0e-a6e4-df2e79364c65 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:46.019260 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e85bf7-0c89-4c0e-a6e4-df2e79364c65 service swift] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:53:46.026289 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c601436-d999-4122-89d2-05d409b74013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:53:46.027149 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c601436-d999-4122-89d2-05d409b74013 service nova] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:46.027626 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c601436-d999-4122-89d2-05d409b74013 service nova] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:46.028068 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c601436-d999-4122-89d2-05d409b74013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:46.030486 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-7c601436-d999-4122-89d2-05d409b74013 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:46.045362 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c601436-d999-4122-89d2-05d409b74013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:53:46.046949 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c601436-d999-4122-89d2-05d409b74013 service nova] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:53:46.071481 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1475/2924] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:53:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:53:52.427540 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=6 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:53:52.958663 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99438160-473f-4698-966e-34b46c1c2a5b None None] REQUEST_METHOD: `POST` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:52.958923 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99438160-473f-4698-966e-34b46c1c2a5b None None] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:52.959201 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99438160-473f-4698-966e-34b46c1c2a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:52.979470 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.auth.core [None req-99438160-473f-4698-966e-34b46c1c2a5b None None] MFA Rules not processed for user `e4ecba03d9254112b343f36ade20754f`. Rule list: `[]` (Enabled: `True`). {{(pid=81483) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:53:52.981016 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-99438160-473f-4698-966e-34b46c1c2a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:53.037101 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1450/2925] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:53:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:53:55.668458 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c601436-d999-4122-89d2-05d409b74013 service nova] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:53:55.675941 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c601436-d999-4122-89d2-05d409b74013 service nova] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:53:55.679937 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1eb6b-c9e3-4a1d-be3a-cb835bc347f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:53:55.680660 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1eb6b-c9e3-4a1d-be3a-cb835bc347f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:53:55.680946 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1eb6b-c9e3-4a1d-be3a-cb835bc347f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:53:55.681333 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1eb6b-c9e3-4a1d-be3a-cb835bc347f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:53:55.683381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-21e1eb6b-c9e3-4a1d-be3a-cb835bc347f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:53:55.694830 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1eb6b-c9e3-4a1d-be3a-cb835bc347f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:53:55.696324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1eb6b-c9e3-4a1d-be3a-cb835bc347f4 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:53:55.706273 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1476/2926] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:53:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:54:03.030457 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=25 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:54:05.702111 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=12 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:56:03.814223 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99438160-473f-4698-966e-34b46c1c2a5b None None] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:56:03.861958 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99438160-473f-4698-966e-34b46c1c2a5b None None] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:56:03.867859 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55909a7b-7dae-42ab-9222-73833bcaa47e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:56:03.868592 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55909a7b-7dae-42ab-9222-73833bcaa47e service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:03.868939 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55909a7b-7dae-42ab-9222-73833bcaa47e service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:03.869371 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55909a7b-7dae-42ab-9222-73833bcaa47e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:03.872216 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-55909a7b-7dae-42ab-9222-73833bcaa47e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:03.907548 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55909a7b-7dae-42ab-9222-73833bcaa47e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:56:03.909176 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55909a7b-7dae-42ab-9222-73833bcaa47e service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:56:03.920770 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1451/2927] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:56:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:56:04.742896 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fe316-aa03-4d00-836b-a1d55bb6c87e None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:04.743427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fe316-aa03-4d00-836b-a1d55bb6c87e None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:04.743427 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fe316-aa03-4d00-836b-a1d55bb6c87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:04.763248 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-149fe316-aa03-4d00-836b-a1d55bb6c87e None None] MFA Rules not processed for user `2dae32845ef94e95a9d8f7c16c73c787`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:56:04.764476 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-149fe316-aa03-4d00-836b-a1d55bb6c87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:04.771623 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1477/2928] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 08:56:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:56:04.814755 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55909a7b-7dae-42ab-9222-73833bcaa47e service nova] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:56:04.826307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55909a7b-7dae-42ab-9222-73833bcaa47e service nova] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:56:04.831382 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b125e4-3ffc-4977-bd4f-e071c23e1ba0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f410e0148c1403098b9b73735425726', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:56:04.832007 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b125e4-3ffc-4977-bd4f-e071c23e1ba0 service swift] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:04.832287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b125e4-3ffc-4977-bd4f-e071c23e1ba0 service swift] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:04.832558 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b125e4-3ffc-4977-bd4f-e071c23e1ba0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:04.840422 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b125e4-3ffc-4977-bd4f-e071c23e1ba0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:56:04.842096 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b125e4-3ffc-4977-bd4f-e071c23e1ba0 service swift] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:56:04.850668 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1452/2929] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 08:56:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 08:56:11.626097 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fe316-aa03-4d00-836b-a1d55bb6c87e None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:56:11.708703 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fe316-aa03-4d00-836b-a1d55bb6c87e None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:56:11.714564 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:56:11.715153 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:11.715449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:11.715727 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] PATH_INFO: `/v3/users/7a41fc47ed5447af88710dd6e74523ce` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:11.716895 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a41fc47ed5447af88710dd6e74523ce)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:56:11.718841 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:56:11.765831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a41fc47ed5447af88710dd6e74523ce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:11.778544 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a41fc47ed5447af88710dd6e74523ce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:11.779211 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a41fc47ed5447af88710dd6e74523ce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:11.786184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a41fc47ed5447af88710dd6e74523ce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:11.786819 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a41fc47ed5447af88710dd6e74523ce'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:11.800034 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1478/2930] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:56:11 2022] DELETE /identity/v3/users/7a41fc47ed5447af88710dd6e74523ce => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:56:11.866871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b125e4-3ffc-4977-bd4f-e071c23e1ba0 service swift] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:56:11.868299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e6b125e4-3ffc-4977-bd4f-e071c23e1ba0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:11.936562 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b125e4-3ffc-4977-bd4f-e071c23e1ba0 service swift] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:56:11.984205 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:56:11.986542 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:11.992950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94363f1-bd59-4185-b64a-039b872de57a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5274740e5d7c4ab1ba9e9781815532c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:56:11.992950 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94363f1-bd59-4185-b64a-039b872de57a service cinder] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:11.993279 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94363f1-bd59-4185-b64a-039b872de57a service cinder] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:11.993503 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94363f1-bd59-4185-b64a-039b872de57a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:11.995260 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-d94363f1-bd59-4185-b64a-039b872de57a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:12.063262 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=100 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:56:12.065764 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94363f1-bd59-4185-b64a-039b872de57a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:56:12.066306 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:56:12.067347 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94363f1-bd59-4185-b64a-039b872de57a service cinder] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:56:12.079415 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1453/2931] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:56:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:56:12.086775 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de74fe-b6c3-4a5f-b539-8cfab4c943ef admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:56:12.091607 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:56:12.092339 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:12.092624 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:12.092899 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] PATH_INFO: `/v3/projects/a5865848bafe480db52b92ab3323645f` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:12.094073 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5865848bafe480db52b92ab3323645f)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:56:12.095999 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:56:12.148143 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5865848bafe480db52b92ab3323645f', 'request_id': 'req-4f41979a-6590-4783-ac95-81906c7054f2'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.155718 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5865848bafe480db52b92ab3323645f', 'request_id': 'req-4f41979a-6590-4783-ac95-81906c7054f2'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.162356 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5865848bafe480db52b92ab3323645f', 'request_id': 'req-4f41979a-6590-4783-ac95-81906c7054f2'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.176201 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] The token cache is being invalidate because project a5865848bafe480db52b92ab3323645f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:56:12.176675 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.177842 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.179368 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1479/2932] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:56:11 2022] DELETE /identity/v3/projects/a5865848bafe480db52b92ab3323645f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:56:12.183531 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94363f1-bd59-4185-b64a-039b872de57a service cinder] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:56:12.198932 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94363f1-bd59-4185-b64a-039b872de57a service cinder] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:56:12.263155 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:56:12.264023 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:12.264361 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:12.264786 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] PATH_INFO: `/v3/users/737518ef022f44c59dd527129048e1ae` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:12.266181 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=737518ef022f44c59dd527129048e1ae)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:56:12.268474 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:56:12.316099 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '737518ef022f44c59dd527129048e1ae'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.322794 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '737518ef022f44c59dd527129048e1ae'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.323452 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '737518ef022f44c59dd527129048e1ae'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.324054 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '737518ef022f44c59dd527129048e1ae'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.331311 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '737518ef022f44c59dd527129048e1ae'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.347866 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1454/2933] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:56:12 2022] DELETE /identity/v3/users/737518ef022f44c59dd527129048e1ae => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:56:12.517313 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:56:12.518846 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:12.532615 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41979a-6590-4783-ac95-81906c7054f2 admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:56:12.595497 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:56:12.596132 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:12.596412 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:12.596686 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e admin admin] PATH_INFO: `/v3/projects/f445710aafeb4a859f155a27d0afb60a` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:12.597837 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f445710aafeb4a859f155a27d0afb60a)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:56:12.600514 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:56:12.648541 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f445710aafeb4a859f155a27d0afb60a', 'request_id': 'req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.654200 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f445710aafeb4a859f155a27d0afb60a', 'request_id': 'req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.660266 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f445710aafeb4a859f155a27d0afb60a', 'request_id': 'req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.674103 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e admin admin] The token cache is being invalidate because project f445710aafeb4a859f155a27d0afb60a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:56:12.674391 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.675031 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-2d532a6f-8e54-4969-ad79-5c4e7a74a25e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:56:12.676465 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1480/2934] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:56:12 2022] DELETE /identity/v3/projects/f445710aafeb4a859f155a27d0afb60a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 08:56:21.524930 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:56:21.526396 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:21.608399 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:56:21.620440 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5abed-94ec-4f49-b6d9-88e952968e08 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:56:21.679218 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba53a8-099b-4e78-bbce-4403d3309f6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:56:21.679887 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba53a8-099b-4e78-bbce-4403d3309f6f service placement] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:21.680168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba53a8-099b-4e78-bbce-4403d3309f6f service placement] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:21.680438 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba53a8-099b-4e78-bbce-4403d3309f6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:21.682213 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-bcba53a8-099b-4e78-bbce-4403d3309f6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:21.693281 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba53a8-099b-4e78-bbce-4403d3309f6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:56:21.694892 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba53a8-099b-4e78-bbce-4403d3309f6f service placement] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:56:21.704802 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1455/2935] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:56:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:56:22.076340 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6318e-aa09-4eaf-8527-62aa421fedac None None] REQUEST_METHOD: `POST` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:22.076861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6318e-aa09-4eaf-8527-62aa421fedac None None] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:22.076861 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6318e-aa09-4eaf-8527-62aa421fedac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:22.086386 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:56:22.099361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.auth.core [None req-44b6318e-aa09-4eaf-8527-62aa421fedac None None] MFA Rules not processed for user `e4ecba03d9254112b343f36ade20754f`. Rule list: `[]` (Enabled: `True`). {{(pid=81482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 08:56:22.101316 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-44b6318e-aa09-4eaf-8527-62aa421fedac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:22.158338 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1481/2936] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 08:56:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 08:56:22.481490 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba53a8-099b-4e78-bbce-4403d3309f6f service placement] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:56:22.482910 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-bcba53a8-099b-4e78-bbce-4403d3309f6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:22.537985 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba53a8-099b-4e78-bbce-4403d3309f6f service placement] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:56:22.542722 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816621b-0ef1-464f-abb8-6bdfab0a0d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:56:22.543393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816621b-0ef1-464f-abb8-6bdfab0a0d4d service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:22.543669 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816621b-0ef1-464f-abb8-6bdfab0a0d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:22.543938 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816621b-0ef1-464f-abb8-6bdfab0a0d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:22.545954 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e816621b-0ef1-464f-abb8-6bdfab0a0d4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:22.557971 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816621b-0ef1-464f-abb8-6bdfab0a0d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:56:22.559409 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816621b-0ef1-464f-abb8-6bdfab0a0d4d service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:56:22.567773 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1456/2937] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:56:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:56:24.014910 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6318e-aa09-4eaf-8527-62aa421fedac None None] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:56:24.016405 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-44b6318e-aa09-4eaf-8527-62aa421fedac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:24.031571 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6318e-aa09-4eaf-8527-62aa421fedac None None] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:56:24.037268 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8facaeb-19dc-4769-9a6c-c7229bc3210e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:56:24.037907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8facaeb-19dc-4769-9a6c-c7229bc3210e service placement] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:24.038189 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8facaeb-19dc-4769-9a6c-c7229bc3210e service placement] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:24.038463 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8facaeb-19dc-4769-9a6c-c7229bc3210e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:24.040194 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c8facaeb-19dc-4769-9a6c-c7229bc3210e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:24.053324 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8facaeb-19dc-4769-9a6c-c7229bc3210e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:56:24.054832 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8facaeb-19dc-4769-9a6c-c7229bc3210e service placement] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:56:24.065384 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1482/2938] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:56:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:56:32.563934 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=59 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:56:34.060631 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=29 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:56:38.338301 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816621b-0ef1-464f-abb8-6bdfab0a0d4d service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:56:38.340753 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-e816621b-0ef1-464f-abb8-6bdfab0a0d4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:38.423412 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816621b-0ef1-464f-abb8-6bdfab0a0d4d service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:56:38.429431 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7f2ef-4c59-42f8-a846-5c34080e2194 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c3c6efc3c154ff89066035cd9583459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:56:38.430033 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7f2ef-4c59-42f8-a846-5c34080e2194 service nova] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:38.430308 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7f2ef-4c59-42f8-a846-5c34080e2194 service nova] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:38.430577 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7f2ef-4c59-42f8-a846-5c34080e2194 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:38.432299 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-edc7f2ef-4c59-42f8-a846-5c34080e2194 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:38.445304 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7f2ef-4c59-42f8-a846-5c34080e2194 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:56:38.446875 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7f2ef-4c59-42f8-a846-5c34080e2194 service nova] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:56:38.457319 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1457/2939] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:56:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:56:41.781416 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8facaeb-19dc-4769-9a6c-c7229bc3210e service placement] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:56:41.800113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8facaeb-19dc-4769-9a6c-c7229bc3210e service placement] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:56:41.804226 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481b795-4cb8-4374-bb16-3acdc1cb7b10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:56:41.804823 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481b795-4cb8-4374-bb16-3acdc1cb7b10 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:56:41.805120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481b795-4cb8-4374-bb16-3acdc1cb7b10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:56:41.805408 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481b795-4cb8-4374-bb16-3acdc1cb7b10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:56:41.807120 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-c481b795-4cb8-4374-bb16-3acdc1cb7b10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:56:41.871219 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481b795-4cb8-4374-bb16-3acdc1cb7b10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:56:41.872656 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481b795-4cb8-4374-bb16-3acdc1cb7b10 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:56:41.882820 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1483/2940] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:56:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:56:48.452806 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=34 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:56:51.877934 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=34 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:58:33.755844 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7f2ef-4c59-42f8-a846-5c34080e2194 service nova] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:58:33.765418 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7f2ef-4c59-42f8-a846-5c34080e2194 service nova] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:58:33.769222 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b673a4-3a78-465e-9428-9c6bc0067e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:58:33.769990 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b673a4-3a78-465e-9428-9c6bc0067e34 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:58:33.770393 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b673a4-3a78-465e-9428-9c6bc0067e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:58:33.770772 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b673a4-3a78-465e-9428-9c6bc0067e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:58:33.773232 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-39b673a4-3a78-465e-9428-9c6bc0067e34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:58:33.811612 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b673a4-3a78-465e-9428-9c6bc0067e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:58:33.813356 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b673a4-3a78-465e-9428-9c6bc0067e34 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:58:33.835450 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1458/2941] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:58:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:58:35.018754 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481b795-4cb8-4374-bb16-3acdc1cb7b10 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:58:35.027899 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481b795-4cb8-4374-bb16-3acdc1cb7b10 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:58:35.031521 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ac7f5-fd25-4147-946e-f7708d8ac342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:58:35.032113 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ac7f5-fd25-4147-946e-f7708d8ac342 service neutron] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:58:35.032392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ac7f5-fd25-4147-946e-f7708d8ac342 service neutron] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:58:35.032664 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ac7f5-fd25-4147-946e-f7708d8ac342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:58:35.034410 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-279ac7f5-fd25-4147-946e-f7708d8ac342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:58:35.048581 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ac7f5-fd25-4147-946e-f7708d8ac342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:58:35.050050 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ac7f5-fd25-4147-946e-f7708d8ac342 service neutron] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:58:35.060892 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1484/2942] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:58:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:58:43.832265 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=18 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:58:45.056036 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=6 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:58:45.864607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b673a4-3a78-465e-9428-9c6bc0067e34 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:58:45.866235 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-39b673a4-3a78-465e-9428-9c6bc0067e34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:58:45.889605 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b673a4-3a78-465e-9428-9c6bc0067e34 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:58:45.893283 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd24c8-6372-4bfd-8db1-18f42dcbd763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736efafe9fc94b45bc75566b1696c498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:58:45.893880 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd24c8-6372-4bfd-8db1-18f42dcbd763 service neutron] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:58:45.894157 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd24c8-6372-4bfd-8db1-18f42dcbd763 service neutron] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:58:45.894427 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd24c8-6372-4bfd-8db1-18f42dcbd763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:58:45.896112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-a2bd24c8-6372-4bfd-8db1-18f42dcbd763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:58:45.958625 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd24c8-6372-4bfd-8db1-18f42dcbd763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:58:45.960674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd24c8-6372-4bfd-8db1-18f42dcbd763 service neutron] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:58:45.970248 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1459/2943] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:58:48.980701 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ac7f5-fd25-4147-946e-f7708d8ac342 service neutron] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:58:48.992767 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ac7f5-fd25-4147-946e-f7708d8ac342 service neutron] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:58:48.998667 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c616-c7a0-4869-8501-89ca16e48fa2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:58:48.999275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c616-c7a0-4869-8501-89ca16e48fa2 service placement] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:58:48.999563 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c616-c7a0-4869-8501-89ca16e48fa2 service placement] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:58:48.999836 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c616-c7a0-4869-8501-89ca16e48fa2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:58:49.001566 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e362c616-c7a0-4869-8501-89ca16e48fa2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:58:49.013907 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362c616-c7a0-4869-8501-89ca16e48fa2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:58:49.015467 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362c616-c7a0-4869-8501-89ca16e48fa2 service placement] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:58:49.027361 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1485/2944] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:58:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:58:51.796287 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd24c8-6372-4bfd-8db1-18f42dcbd763 service neutron] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:58:51.808812 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd24c8-6372-4bfd-8db1-18f42dcbd763 service neutron] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:58:51.814674 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832138f-e644-4ccb-bcb2-5d43fe43b562 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:58:51.815367 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832138f-e644-4ccb-bcb2-5d43fe43b562 service placement] REQUEST_METHOD: `GET` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:58:51.816046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832138f-e644-4ccb-bcb2-5d43fe43b562 service placement] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:58:51.816046 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832138f-e644-4ccb-bcb2-5d43fe43b562 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:58:51.818719 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-6832138f-e644-4ccb-bcb2-5d43fe43b562 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:58:51.832401 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832138f-e644-4ccb-bcb2-5d43fe43b562 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:58:51.834097 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832138f-e644-4ccb-bcb2-5d43fe43b562 service placement] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:58:51.848112 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1460/2945] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 08:58:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 08:58:51.880430 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c616-c7a0-4869-8501-89ca16e48fa2 service placement] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:58:51.882479 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-e362c616-c7a0-4869-8501-89ca16e48fa2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:58:51.897184 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c616-c7a0-4869-8501-89ca16e48fa2 service placement] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:58:51.902739 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:58:51.903381 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:58:51.903638 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:58:51.903908 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] PATH_INFO: `/v3/users/e4ecba03d9254112b343f36ade20754f` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:58:51.905014 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ecba03d9254112b343f36ade20754f)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:58:51.906935 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:58:51.953803 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ecba03d9254112b343f36ade20754f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:51.970864 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ecba03d9254112b343f36ade20754f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:51.971597 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ecba03d9254112b343f36ade20754f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:51.982612 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ecba03d9254112b343f36ade20754f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:51.983275 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ecba03d9254112b343f36ade20754f'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:51.996308 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1486/2946] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:58:51 2022] DELETE /identity/v3/users/e4ecba03d9254112b343f36ade20754f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:58:52.447533 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832138f-e644-4ccb-bcb2-5d43fe43b562 service placement] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:58:52.458307 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832138f-e644-4ccb-bcb2-5d43fe43b562 service placement] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:58:52.515643 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:58:52.516277 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:58:52.516555 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:58:52.516852 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] PATH_INFO: `/v3/projects/ba777b9726e944a881c0953e605f7477` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:58:52.518067 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba777b9726e944a881c0953e605f7477)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:58:52.519977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:58:52.567928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba777b9726e944a881c0953e605f7477', 'request_id': 'req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:52.573871 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba777b9726e944a881c0953e605f7477', 'request_id': 'req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:52.581928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba777b9726e944a881c0953e605f7477', 'request_id': 'req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:52.591118 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:58:52.595807 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] The token cache is being invalidate because project ba777b9726e944a881c0953e605f7477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:58:52.596188 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:52.597212 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:52.599583 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1461/2947] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:58:52 2022] DELETE /identity/v3/projects/ba777b9726e944a881c0953e605f7477 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:58:52.604047 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:58:52.605572 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:58:52.619229 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfcfa8-a1f8-41ba-b086-ccc0a6876cca admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:58:52.674831 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:58:52.675443 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:58:52.675719 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:58:52.675987 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] PATH_INFO: `/v3/users/ec17b7d0b9764ab1bc198249c3ffe571` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:58:52.677118 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec17b7d0b9764ab1bc198249c3ffe571)` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:58:52.679658 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:58:52.724100 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec17b7d0b9764ab1bc198249c3ffe571'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:52.734449 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:58:52.737056 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec17b7d0b9764ab1bc198249c3ffe571'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:52.737791 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec17b7d0b9764ab1bc198249c3ffe571'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:52.746595 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec17b7d0b9764ab1bc198249c3ffe571'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:52.747237 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.notifications [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec17b7d0b9764ab1bc198249c3ffe571'} {{(pid=81482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:52.761185 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1487/2948] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 08:58:52 2022] DELETE /identity/v3/users/ec17b7d0b9764ab1bc198249c3ffe571 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:58:52.926607 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] Authenticating user token {{(pid=81483) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 08:58:52.928168 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.fernet_utils [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81483) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 08:58:52.944170 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cd12d-8a7a-456d-afb2-0b0bc3402e13 admin admin] Validating token access rules against request {{(pid=81483) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 08:58:53.003535 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06286873-b2f7-4670-ad2f-5a355f3f81b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3300b0df5e5741c8b0d940d0202bb604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c189a0d1d6b0443ba4af2cd3c46f5c43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81483) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 08:58:53.004098 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06286873-b2f7-4670-ad2f-5a355f3f81b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 08:58:53.004380 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06286873-b2f7-4670-ad2f-5a355f3f81b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 08:58:53.004658 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06286873-b2f7-4670-ad2f-5a355f3f81b2 admin admin] PATH_INFO: `/v3/projects/f62014b9d5b9457ca4e1e6b1d1da0919` {{(pid=81483) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 08:58:53.005797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06286873-b2f7-4670-ad2f-5a355f3f81b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f62014b9d5b9457ca4e1e6b1d1da0919)` {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 08:58:53.007797 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06286873-b2f7-4670-ad2f-5a355f3f81b2 admin admin] RBAC: Authorization granted {{(pid=81483) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 08:58:53.054395 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-06286873-b2f7-4670-ad2f-5a355f3f81b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f62014b9d5b9457ca4e1e6b1d1da0919', 'request_id': 'req-06286873-b2f7-4670-ad2f-5a355f3f81b2'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:53.059788 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-06286873-b2f7-4670-ad2f-5a355f3f81b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f62014b9d5b9457ca4e1e6b1d1da0919', 'request_id': 'req-06286873-b2f7-4670-ad2f-5a355f3f81b2'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:53.067928 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-06286873-b2f7-4670-ad2f-5a355f3f81b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f62014b9d5b9457ca4e1e6b1d1da0919', 'request_id': 'req-06286873-b2f7-4670-ad2f-5a355f3f81b2'} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:53.078963 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-06286873-b2f7-4670-ad2f-5a355f3f81b2 admin admin] The token cache is being invalidate because project f62014b9d5b9457ca4e1e6b1d1da0919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81483) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 08:58:53.079255 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-06286873-b2f7-4670-ad2f-5a355f3f81b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:53.079876 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG keystone.notifications [None req-06286873-b2f7-4670-ad2f-5a355f3f81b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81483) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 08:58:53.081411 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: [pid: 81483|app: 0|req: 1462/2949] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 08:58:52 2022] DELETE /identity/v3/projects/f62014b9d5b9457ca4e1e6b1d1da0919 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 08:59:02.755836 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 08:59:03.075977 dv-cinder-844563-8-iscsi devstack@keystone.service[81483]: DEBUG dbcounter [-] [81483] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81483) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 09:01:25.511528 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] Authenticating user token {{(pid=81482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 09:01:25.512988 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 09:01:25.583794 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10bd5e-ac42-4691-946e-78626e2c62fd admin admin] Validating token access rules against request {{(pid=81482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 09:01:25.643116 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61400d2d-52ba-4f4e-afe6-6d166afad307 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b40ee263df74e58bf3f958c01eb742f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca4d4801f2f40419f6881984fbbfbcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 09:01:25.643721 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61400d2d-52ba-4f4e-afe6-6d166afad307 service placement] REQUEST_METHOD: `GET` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 09:01:25.644000 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61400d2d-52ba-4f4e-afe6-6d166afad307 service placement] SCRIPT_NAME: `/identity` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 09:01:25.644274 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61400d2d-52ba-4f4e-afe6-6d166afad307 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 09:01:25.646002 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.fernet_utils [None req-61400d2d-52ba-4f4e-afe6-6d166afad307 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 09:01:25.657707 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61400d2d-52ba-4f4e-afe6-6d166afad307 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 09:01:25.659761 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61400d2d-52ba-4f4e-afe6-6d166afad307 service placement] RBAC: Authorization granted {{(pid=81482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 09:01:25.669966 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: [pid: 81482|app: 0|req: 1488/2950] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 09:01:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 09:01:35.665392 dv-cinder-844563-8-iscsi devstack@keystone.service[81482]: DEBUG dbcounter [-] [81482] Writing DB stats keystone:SELECT=58 {{(pid=81482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}